- Family and Friends
Archive for category N3RD
Having fun with hardware is a good way to learn about the machines which soon will become our new overlords. With this pretty interesting presentation you can dive deep into what a CPU does and how it can be exploited to run code by not running it.
“Trust Analysis, i.e. determining that a system will not execute some class of computations, typically assumes that all computation is captured by an instruction trace. We show that powerful computation on x86 processors is possible without executing any CPU instructions. We demonstrate a Turing-complete execution environment driven solely by the IA32 architecture’s interrupt handling and memory translation tables, in which the processor is trapped in a series of page faults and double faults, without ever successfully dispatching any instructions. The “hard-wired” logic of handling these faults is used to perform arithmetic and logic primitives, as well as memory reads and writes. This mechanism can also perform branches and loops if the memory is set up and mapped just right. We discuss the lessons of this execution model for future trustworthy architectures.”
Since I’ve become sort of an iOS developer lately I had my fair share of WWDC recordings to get started with this whole CocoaTouch and Objective-C development stuff.
Now a tool that is pretty handy is a this website that offers a full-text transcript search of all WWDC recordings. Awesome!
SDR – or Software Defined Radio is relatively cheap and fun way to dive deeper into radio communication.
“Software-defined radio (SDR) is a radio communication system where components that have been typically implemented in hardware (e.g. mixers, filters, amplifiers, modulators/demodulators, detectors, etc.) are instead implemented by means of software on a personal computer or embedded system. While the concept of SDR is not new, the rapidly evolving capabilities of digital electronics render practical many processes which used to be only theoretically possible.” (Wikipedia)
So with cheap hardware it’s possible to receive radio transmissions on all sorts of frequencies and modulations. Since everything after the actual “receiving stuff”-phase happens in software the things you can do are sort of limitless.
Now what about the relatively cheap factor? – The hardware you’re going to need to start with this is a DVB-T USB stick widely available for about 25 Euro. The important feature you’re going to look for is that it comes with a Realtek RTL2832U chip.
“The RTL2832U is a high-performance DVB-T COFDM demodulator that supports a USB 2.0 interface. The RTL2832U complies with NorDig Unified 1.0.3, D-Book 5.0, and EN300 744 (ETSI Specification). It supports 2K or 8K mode with 6, 7, and 8MHz bandwidth. Modulation parameters, e.g., code rate, and guard interval, are automatically detected.
The RTL2832U supports tuners at IF (Intermediate Frequency, 36.125MHz), low-IF (4.57MHz), or Zero-IF output using a 28.8MHz crystal, and includes FM/DAB/DAB+ Radio Support. Embedded with an advanced ADC (Analog-to-Digital Converter), the RTL2832U features high stability in portable reception.” (RealTek)
You’ll find this chip in all sorts of cheap DVB-T USB sticks like this one:
To use the hardware directly you can use open source software which comes pre-packaged with several important/widely used demodulator moduls like AM/FM. Gqrx SDR is available for all sorts of operating systems and comes with a nice user interface to control your SDR hardware.
The neat idea about SDR is that you, depending on the capabilities of your SDR hardware, are not only tuned into one specific frequency but a whole spectrum several Mhz wide. With my device I get roughly a full 2 Mhz wide spectrum out of the device allowing me to see several FM stations on one spectrum diagram and tune into them individually using the demodulators:
The above screenshot shows the OS X version of Gqrx tuned into an FM station. You can clearly see the 3 stations that I can receive in that Mhz range. One very strong signal, one very weak and one sort of in the middle. By just clicking there the SDR tool decodes this portion of the data stream / spectrum and you can listen to a FM radio station.
Of course – since those DVB-T sticks come with a wide spectrum useable – mine comes with an Elonics E4000 tuner which allows me to receive – more or less useable – 53 Mhz to 2188 Mhz (with a gap from 1095 to 1248 Mhz).
Whatever your hardware can do can be tested by using the rtl_test tool:
root@berry:~# rtl_test -t
Found 1 device(s):
0: Terratec T Stick PLUS
Using device 0: Terratec T Stick PLUS
Found Elonics E4000 tuner
Supported gain values (14): -1.0 1.5 4.0 6.5 9.0 11.5 14.0 16.5 19.0 21.5 24.0 29.0 34.0 42.0
Benchmarking E4000 PLL…
[E4K] PLL not locked for 52000000 Hz!
[E4K] PLL not locked for 2189000000 Hz!
[E4K] PLL not locked for 1095000000 Hz!
[E4K] PLL not locked for 1248000000 Hz!
E4K range: 53 to 2188 MHz
E4K L-band gap: 1095 to 1248 MHz
Interestingly when you plug the USB stick into an Raspberry Pi and you follow some instructions you can use the Raspberry Pi as an SDR server allowing you to place it on the attic while still sitting comfortably at your computer downstairs to have better reception.
If you want to upgrade your experience with more professional hardware – and in fact if you got a sender license – you can take a look at the HackRF project which currently is creating a highly sophisticated SDR hardware+software solution:
Source 1: http://www.realtek.com.tw/products/productsView.aspx?Langid=1&PFid=35&Level=4&Conn=3&ProdID=257
Source 2: http://gqrx.dk/
Source 3: www.hamradioscience.com/raspberry-pi-as-remote-server-for-rtl2832u-sdr/
Source 4: http://ossmann.blogspot.de/2012/06/introducing-hackrf.html
Source 5: https://github.com/mossmann/hackrf
If you want to interface with the publicly available instance of the miataru server you can use the URL: http://service.miataru.com. This URL also is pre-configured with the iOS client that got recently available in the AppStore.
I started working on a Node.js project and so far it’s a quite satisfying experience. But what is Node.js?
There are a lot of things that are approached differently in Node. One of which is how you deal with code and debuggings.
I come from a world of fully integrated development environments. I had that for C#, it’s there for Java, it’s even there for Objective-C.
So it’s a bit like a toolbox you are supposed to put together yourself. And in this article I want to describe how a 2-week-beginners development environment for Node looks like. If you got anything to improve or add – go ahead, leave a comment!
GIT! I am using GitX and command line git to work with the source control. Nothing special really.
You got a lot of options here. May it be the awesome Sublime Text 2 or Eclipse or NetBeans. I chose Coda 2 since I already got it and was using it for my humble web development intermezzos. It’s awesome and if you’re on Mac you should give it a try!
Now things are getting interesting. To debug Node.js applications you have a lot of options from which a lot of them works quite good. Unfortunately I was not able to find the one IDE that provides all in one – great code editing and good debugging. So I chose to use a stand-alone debugging solution that does the trick in the best way I can think of. It’s called node-inspector and is available on all possible platforms as it seems.
Triggering and Glue
There’s only one thing left right now which is hindering the code hacking and debugging. And it’s the fact that Node.js in it’s default state does not reload changed local code files after it loaded them once. And this means: When you edit something you would have to manually restart Node.js to see the changes you just made in effect. And that’s where a little tool called Supervisor comes into play. It watches the files of your project and kills+restarts Node.js automatically for you and takes care of that bugging restart-cycle. It just works!
If course there are some more things in regards of writing tests. But this is going to be another article.
Source 1: http://nodejs.org
Source 2: http://en.wikipedia.org/wiki/Node.js
Source 3: http://panic.com/coda/
Source 4: https://github.com/node-inspector/node-inspector
Source 5: https://github.com/isaacs/node-supervisor
I am a long time user of Google Latitude – and since I added a Google Latitude Module to h.a.c.s. almost the whole family started using this service. It’s all about tracking your location.
“Latitude has been retired
Google Latitude was retired on August 9th, 2013. Products retired include Google Latitude in Google Maps for Android, Latitude for iPhone, the Latitude API, the public badge, the iGoogle Gadget, and the Latitude website at maps.google.com/latitude.
What does this mean for me?
- You are no longer able to share your location using Latitude….”
We used it for a lot of use cases. If just to know if the other is en-route to a meeting-point or to know if someone arrived safely during a long trip. Or in terms of home automation to let the house know if you are there or somewhere else – for instance to enable or disable the house alarming system or to power up / shutdown the heating if necessary.
After the retirement of Latitude on the 9th of August all those use cases where not doable anymore. Yes there are some tools that do this and that for Location tracking. But when Google Latitude was still active it did not fullfil all use-cases I would have gotten – it was just “good enough”. Now all those subtitutes are not even close to a fraction of the use cases I would have.
Now what? Easy! If nothing works out, you gotta do it yourself!
So I started a new spare-time project I call Miataru. Weird name, eh?
“Miataru or 見当たる is Japanese and means “be found” or “to come across” and it’s meant to be a set of tools to allow the user to track locations and choose how to work with the data as well as how data is stored if at all.” (Miataru.com)
So – this should not be a replacement for anyone for Google Latitude. But the goal is to create a client+server toolset that allows you to cover a lot of use cases around location tracking and the interfacing with other software like home automation.
So expect some articles here about all the funny things and learnings about NodeJS and Objective-C / iOS development.
Some quick words to all you readers:
If you want to participate in an open source project in NodeJS and Mobile devices you’re invited to join anytime!
I’ve added Alarming to hacs a while ago and I’ve now extended the built-in SMS gateway providers with the german telekom services called “Global SMS API”.
This API is offered through the Telekom own portal called developer garden and is as easy to use as it can possibly be. You only need to set-up the account with developergarden and after less than 5 minutes you can send and receive SMS and do a lot more. They got APIs for nearly everything you possible want to do … fancy some “talk to your house”-action? Would be easy to integrate into h.a.c.s. using their Speech2Text APIs.
They have a short video showing how to set it all up:http://www.youtube.com/watch?v=caRSafzMDK0
So I’ve added the SMS-send capabilities to the hacs internal alarming system with it’s own JSON configuration file looking like this:
And this simple piece of configuration leads to SMS getting sent out as soon as – in this example – a window opens:
Before the Telekom Global SMS API I’ve used a different provider (SMS77) but since the delivery times of this provider varied like crazy (everything from 30 seconds to 5 minutes) and the provider had a lot of downtimes my thought was to give the market leader a try.
So now here it is – integrated. Get the source here.
Obviously it’s impossible for Apple to fix that quite annoying bug in their operating system that leads to double/tripple/… program entries in the “Open with…” menu. Everytime an application is updated it adds a new entry but does not remove the old one.
This makes your open-width menu look like this:
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user;killall Finder
This simple command will kill the double/tripple/… entries and restarts your finder.app to make the change visible. Your “Open with…” menu should now only show singular entries per application:
“Hyper-lapse photography – a technique combining time-lapse and sweeping camera movements typically focused on a point-of-interest – has been a growing trend on video sites. It’s not hard to find stunning examples on Vimeo. Creating them requires precision and many hours stitching together photos taken from carefully mapped locations. We aimed at making the process simpler by using Google Street View as an aid, but quickly discovered that it could be used as the source material. It worked so well, we decided to design a very usable UI around our engine and release Google Street View Hyperlapse.“
Die verfügbaren IPv4 Adressen neigen sich dem Ende und IPv6 wird kommen. Da gibt es keinen Zweifel! Dieses Weblog beispielsweise ist seit über zwei Jahren nativ über IPv6 erreichbar. Nun wird es mit jedem Monat der ins Land geht immer ‘brenzliger’ und dementsprechend wichtig ist der Schritt unter anderem auch für die öffentliche Verwaltung. Interessante Einblicke gibt dieses umfangreiche Dokument:
“Seit den Anfangstagen des Internets wird zur Übertragung der Daten das Internet Protokoll in der Version 4 (IPv4) verwendet. Heute wird dieses Protokoll überall verwendet auch in den internen Netzen von Behörden und Organisationen. Das Internet und alle Netze, welche IPv4 heute verwenden, stehen vor einem tiefgreifenden technischen Wandel, denn es ist zwingend für alle zum Nachfolger IPv6 zu wechseln.
Auf die oft gestellte Frage, welche wesentlichen Faktoren eine Migration zu IPv6 vorantreiben, gibt es zwei zentrale Antworten:
- Es gibt einen Migrationszwang der auf die jetzt schon (in Asien) nicht mehr verfügbaren IPv4-Adressen zurückführen ist.
- Mit dem steigenden Adressbedarf für alle Klein- und Großgeräte, vom Sensor über Smartphones bis zur Waschmaschine, die über IP-Netze kommunizieren müssen verschärft sich das Problem der zur Neige gegangenen IPv4-Adressräume. Das Zusammenkommen beider Faktoren beschleunigt den Antrieb zur IPv6-Migration.
Es wird in Zukunft viele Geräte geben, die nur noch über eine IPv6-Adresse anstatt einer IPv4-Adresse verfügen werden und nur über diese erreichbar sind. Schon heute ist bei den aktuellsten Betriebssystemversionen IPv6 nicht mehr ohne Einschränkungen deaktivierbar. Restliche IPv4-Adressen wird man bei Providern gegen entsprechende Gebühren noch mieten können. Bei einem Providerwechsel im Kontext einer Neuausschreibung von Dienstleistungen wird man diese jedoch nicht mehr ‘mitnehmen’ können. Damit bedeutet eine Migration zu IPv6 nicht nur die garantierte Verfügbarkeit ausreichend vieler IP-Adressen, sondern stellt auch die Erreichbarkeit eigener Dienstleistungen für die Zukunft sicher ohne von einem Anbieter abhängig zu sein.”
Last year in June I wrote about the concept of a ubiquitous status display of the business in every office. Especially for development and operations it’s pretty important to have important measurements, status codes and project information in front of them all the time.
Back then I already wrote about the Panic status board which gives a great looking example of a status display. Now there is a software from the company Panic which offers anyone the ability to create such a status board. It’s for iOS and looks awesome!
The number is 27!
Right now there are 27 different missions ongoing to explore our solar system. A high number for something that is not part of our daily news cycle. Those missions currently concentrate on the sun, mars, mercury, venus, the earth moon and some asteriods.
Did you ever start a horde of virtual machines and a complicated vm-only network set-up just to simulate a medium complex network and the interaction of nodes in that network? Well that’s a tiresome, error-prone and labour intensive process. Fear no more, there’s a tool to the rescue.
“Mininet creates a realistic virtual network, running real kernel, switch and application code, on a single machine (VM, cloud or native), in seconds, with a single command:”
“Because you can easily interact with your network using the Mininet CLI (and API), customize it, share it with others, or deploy it on real hardware, Mininet is useful for development, teaching, and research. Mininet is also a great way to develop, share, and experiment with OpenFlow and Software-Defined Networking systems.
For just shy of 2 years I am a fan of whisky. After I got the hang of the processes, tastes and smells around this spirit I started collecting them – collecting to drink them eventually.
Now there are a number of shops you can buy good quality whisky from anywhere in the world. One of which happens to be located in germany. This shop is not only offering a huge choice but also a cross-sellers dream: tasting and explanation videos beneath many of the whiskys in which a very talented Mr. Horst Lüning tastes and explains all things whisky.
Now this shop hosts all videos on YouTube. Since I am a big fan of podcasting and internet based entertainment it’s a great thing that because if my little tool called “YouTubeFeast” all new episodes and tasting videos get downloaded automatically. Till today this way I’ve got well over 650 whisky tasting and explanation videos downloaded.
As a matter of fact this is a really entertaining and educating series I even would pay to get access to. But that aside every video which got automatically downloaded usually looks like this (german audio):
As you can see there’s a short intro (8 seconds) and an outro (29 seconds) which every single video starts and ends with. Under normal circumstances there are two occasions when I have those videos played.
- When I want to look for a particular whisky and get an overview of how it’s going to be like.
- For over 12 years I happen to have a “nights playlist” – a playlist of things that are played back during the night – every night. For this it’s important that it’s mainly speech, very normalized audio and of course it needs to be interesting.
So for the second reason it’s important that there are not too many audio bumps and breaks. Unfortunately as much as I like the intro and outro music it’s actually very bass heavy and as such sleep interrupting sometimes… So just like when a good newmake spirit is distilled the start and end run need to be separated by the heart that makes up the spirit.
Every 4-6 months I take all newly added videos and cut them down and add them to the nights playlist folders. The process is like this:
- Rename them: Remove the following things from the filenames
“Whiskey Verkostung – “, “Whiskey Likör Verkostung “, “Whiskygläser “, “Whisky-Verkostung – “, “Whisky Vorstellung “, “Whisky Verkostung “, “Whisky Verkosten “, “Whisky Tasting – “, “Whisky Tasting “, “Whisky Likör Verkostung “, “Whiskey-Verkostung – “, “Whiskey Verkostung “, “Whiskey Tasting – “, “- ”
To rename the files I am usually using the freeware tool Rename Master – it’s awesome!
- Cut the intro away.
This best done with a simple ffmpeg command:
ffmpeg -i $inputfile -ss 00:00:08.0 -acodec copy -vcodec copy $output
- Cut the outro away.
Using a little shell script it’s fairly easy to first get the full length of each video file and then using another tool to substract 29 seconds from each length and cut the heart out until that length is reached.
To get the length the following short line is doing a great job:
ffmpeg -i “$1” 2>&1 |grep Duration | cut -d ‘ ‘ -f 4 | sed s/,//
In order to then cut the video before the outro starts it basically is a another call to ffmpeg:
ffmpeg -i $infile -t $calculatedlength -acodec copy -vcodec copy $output
That way you get just the tasting videos without intro and outro – ready to be enjoyed. For the end of this article I want to stress the fact how awesome I think those whisky videos from Mr. Lüning are. It’s awesome to watch and learn. I hope that those videos will be available for more years to come! Cheers!
Ever wondered what earth looks live from orbit? Well there are several cameras in the ISS which stream (when ISS is over daylight-territory) an live image to earth for you to see.
If you want to know where the International Space Station currently is you can always click here.
It’s becoming a fashion lately to release the source code of older but legendary commercial products to the public. Now Adobe decided to gift the source code of their flagship product Photoshop in it’s first version from 1990 to the Computer History Museum.
“That first version of Photoshop was written primarily in Pascal for the Apple Macintosh, with some machine language for the underlying Motorola 68000 microprocessor where execution efficiency was important. It wasn’t the effort of a huge team. Thomas said, “For version 1, I was the only engineer, and for version 2, we had two engineers.” While Thomas worked on the base application program, John wrote many of the image-processing plug-ins.”
Since my wife started working as a photographer on a daily basis the daily routine of getting all the pictures off the camera after a long day filled with photo shootings got her bored quickly.
Since we got some RaspberryPis to spare I gave it a try and created a small script which when the Pi gets powered on automatically copies all contents of the attached SD card to the houses storage server. Easy as Pi(e) – so to speak.
So this is now an automated process for a couple of weeks – she comes home, get’s all batteries to their chargers, drops the sd cards into the reader and poweres on the Pi. After it copied everything successfully the Pi sends an eMail with a summary report of what has been done. So far so good – everything is on our backuped storage server then.
Now the problem was that she often does not immediately starts working on the pictures. But she wants to take a closer look without the need to sit in front of a big monitor – like taking a look at her iPad in the kitchen while drinking coffee.
So what we need was a tool that does this:
- take a folder (the automated import folder) and get all images in there, order them by day
- display an overview per day of all pictures taken
- allow to see the fullsized picture if necessary
- work on any mobile or stationary device in the household – preferably html5 responsive design gallery
- it should be fast because commonly over 200 pictures are done per day
- it should be opensource because – well opensource is great – and probably we would need to tweak things a bit
Since I did not find anything near what we had in mind I sat down this afternoon and wrote a tool myself. It’s opensourced and available for you to play with it. Here’s a short description what it does:
It’s pretty fast because it’s not actively resizing the images – instead it’s taking the thumbnail picture from the original jpg file which the camera placed there during storing the picture. It’s got some caching and can be run on any operating system where mono / .net is available – which is probably anything – even the RaspberryPi.
“Say it with pictures. Describe your feelings about your everyday sysadmin interactions.”
Everybody knows ZIP files. It’s what comes out when you compress something on windows and on OS X. It’s the commonly used format to store and exchange compressed data.
Now there’s a lot of things you can do when you know file formats, especially those with many algorithms involved, inside out. There is a lot of text explaining the ZIP file format, like this one.
The report for 2012 is in! Since 2008 Jehiah Czebotar is monitoring his daily life and he is compiling a report from that data for everyone to read. He self says that this is a hat tip to Nicholas Felton who himself is releasing beautiful yearly reports of statistics around his life.
I am a fan of those nice graphics and statistics about the life. It really gives you insights that you wouldn’t be able to get otherwise. Especially with my own home automation and self-monitoring ambitions it’s quite a load of new ideas coming in from these nice graphics.
If you need data to fill your brand new (graph) database, go ahead, there’s something to load:
“KONECT (the Koblenz Network Collection) is a project to collect large network datasets of all types in order to perform research in network science and related fields, collected by the Institute of Web Science and Technologies at the University of Koblenz–Landau. KONECT contains over a hundred network datasets of various types, including directed, undirected, bipartite, weighted, unweighted, signed and rating networks. The networks of KONECT are collected from many diverse areas such as social networks, hyperlink networks, authorship networks, physical networks, interaction networks and communication networks. The KONECT project has developed network analysis tools which are used to compute network statistics, to draw plots and to implement various link prediction algorithms. The result of these analyses are presented on these pages. Whenever we are allowed to do so, we provide a download of the networks.”
KONECT currently holds 157 networks, of which
Source 1: http://konect.uni-koblenz.de/
If you can stand a little bit of cursing and bad words and if you’re a developer. You should give this site a visit. The commit logs from last night speak for themselves:
It’s been a habbit to ID software to release the source code of their previous games and game engines as open source when time is due. That’s what happened with Doom 3 as well. Since beautiful code appears to a lot of developers it’s just a logical step to analyse the Doom 3 source code with the beauty-aspects in mind.
Now there are two very good examples of such analysis.
Source 1: http://kotaku.com/5975610/the-exceptional-beauty-of-doom-3s-source-code
Source 2: ftp://ftp.idsoftware.com/idstuff/doom3/source/CodeStyleConventions.doc
Source 3: http://fabiensanglard.net/doom3/index.php
Source 4: https://github.com/TTimo/doom3.gpl
Usually the actors that allow you to switch power on/off and who measure power usage use the 434Mhz or 868Mhz wireless bands to communicate with their base station. Now the german manufacturer AVM came up with a solution that allows you to switch on/off (with an actual button on the device itself and wireless!) and to measure the power consumption of the devices connected to it.
The unspectacular it looks the spectacular are the features:
- switch up to 2300 watts / 10 ampere
- use different predefined settings to switch on/off or even use Google Calendar to tell it when to switch
- measure the energy consumption of connected devices
- it uses the european DECT standard to communicate with a Fritz!Box base station (which is a requirement)
For around 50 Euro it’s quite an investment but maybe I’ll give it a shot – especially the measurement functionality sounds great. Since I do not have one yet I don’t know anything about how to access it through third party software (h.a.c.s.?)
Source 1: www.avm.de/de/News/artikel/2013/start_fritz_dect_200.html
Source 2: www.avm.de/de/Produkte/Smart_Home/FRITZDECT_200/index.php
Source 3: en.wikipedia.org/wiki/Digital_Enhanced_Cordless_Telecommunications
Behold the beauty of the earth by night from orbit:
“0 A.D. (pronounced “zero-ey-dee”) is a free, open-source, historical Real Time Strategy (RTS) game currently under development by Wildfire Games, a global group of volunteer game developers. As the leader of an ancient civilization, you must gather the resources you need to raise a military force and dominate your enemies.”
Source 1: http://play0ad.com/
And once again some smart people put their heads together and came up with something that will revolutionize your world. Well it’s ‘just’ home automation but indeed it looks very very promising. Especially the human-machine interface through speech recognition. First of all let’s start with a short introductory video:
“CastleOS is an integrated software suite for controlling the automation equipment in your home – an operating system for your castle, if you will. The first piece of the suite is what we call the “Core Service” – it acts as the central controller for the whole system. This runs on any relatively recent Windows computer (or more specifically, the computer that has an Insteon PLM or USB stick plugged in to it), and creates a network connection to both your home automation devices, and the second piece of the integrated suite – the remote access apps like the HTML5 app, Kinect voice control app, and future Android/iOS apps.” (from the CastleOS page)
So it’s said to be an all-in-one system that controls power-outlets and devices through it’s core service and offering the option to add Kinect based speech recognition to say things like “Computer, Lights!”.
Unfortunately it comes with quite high and hard requirements when it comes to hardware it’s compatible with. A kinect possible exists in your household but I doubt that you got the Insteon hardware to control out devices with.
That seems to be the main problem of all current home automation solutions – you just have to have the according hardware to use them. It’s not quite possible to use anything and everything in a standardized way. Maybe it’s time to have a “home plug’n’play” specification set-up for all hard- and software vendors to follow?
Source 1: http://www.castleos.com
So first a small video to get an idea what I am implementing right now:
- to draw svg in a human-controllable
- to draw those nice gauges
- an animated HTML5 canvas odometer
I plan to add a lot more – like for swiping gestures. So this will be – just like h.a.c.s – a continuous project. Since I switched to OS X entirely at home I use the great Coda2 to write and debug the code. It helps a lot to have two browser set-up because for some reason I still not feel that well with the WebKit Web Inspector.
Another great feature of Coda2 is the AirPreview – which means it will preview your current page in the editor on an iOS device running DietCoda – oh how I love those automations.
Every year between Christmas and New Years the hackers of the (mostly european) world gather for the Chaos Communication Congress. This year for the 29th time. The 29c3 takes place where it all started – in Hamburg. This years subtitle is:
Since the reports are already in that the fairydust has landed successfully in Hamburg there’s even a proof picture for it:
Since FeM is already preparing it will be great to ‘attend’ the congress via live streams of all lectures.
Source 1: https://events.ccc.de/congress/2012/wiki/Main_Page
Source 2: http://blog.fem.tu-ilmenau.de/archives/836-Reisetagebuch-Mal-kurz-Hamburg.html
The other day I found out that there is an actual hackerspace in Bamberg – the city where I work and live nearby. For some strange reason it never occurred to me to search for an hackerspace nearby. But now since the 29c3 is at the gates I found them on the “Congress everywhere” pages (beware, it’s having a hard time right now).
Since I just found it and christmas duties take their toll I wasn’t able to go by and talk to the people there in person – i’ve just contacted them over their IRC channel (#backspace on freenode). Eventually I will have time to visit them and I’ll have a report up here then.
For the time being enjoy their website and the projects they already did. Apparently there are some very interesting LED lighting experiments.
Wikipedia describes latency this way:
“Latency is a measure of time delay experienced in a system, the precise definition of which depends on the system and the time being measured. In communications, the lower limit of latency is determined by the medium being used for communications. In reliable two-way communication systems, latency limits the maximum rate that information can be transmitted, as there is often a limit on the amount of information that is “in-flight” at any one moment. In the field of human-machine interaction, perceptible latency has a strong effect on user satisfaction and usability.” (Wikipedia)
Given that it’s quite important for any developer to know his numbers. Since latency has a huge impact on how software should be architected it’s important to keep that in mind:
I am using some Raspberry Pis to monitor the areas around the house. Mainly because it’s awesome to see how many animals are roaming around in your garden throughout the day. On the Pi I am using the current Debian image and motion to interface with an USB webcam.
Now I wanted to include sensory data into the webcam images – like the current temperature. The nice thing about h.a.c.s. is that it can deliver every sensors data in nice and easy to use JSON. The only challenge now is to get the number into motion.
First of all I need to get the URL together where I can access sensor data for the right sensor. In this case it’s the sensor called “Schuppen” – an outdoor sensor measuring the current temperature around the house.
Now there is an easy way to ‘feed’ data into a running motion instance. Motion offers a control port and allows to set the text_left and text_right properties. Doing a simple GET request there allows us to set the text to – in this example – “remote-controlled-text”:
So – that’s how the text is set – now how to get the temperature value, and just that, out of the JSON response of h.a.c.s.? Easy – use jsawk!
With all that a very small shell script is quickly hacked:
If you want to copy that into your editor, here’s the code:
#!/bin/bashTEMPERATURE=`curl -s 'http://hacs/data/sensor?name=Schuppen&type=temperature&lastentry=true' | jsawk 'return this.data'` curl -s 'http://localhost:8080/0/config/set?text_left='$TEMPERATURE
Localhost port 8080 is the port and adress of the motion control server .
To have the webcam updated regularly, I added it to crontab and from now on the current temperature is in every webcam image – hurray!
Source 1: motion
Source 2: jsawk
So after I wrote about the great fileserver expansion plans I thought I give a short update on the progress:
SPAUN or Semantic Pointer Architecture Unified Network is a promising next step in the pursuit to simulate a human brain. Built upon the Nengo Neural Simulator scientists at the University in Waterloo/Ontario were able to report on their first break-through results.
In 2013 there will be a book from Oxford University press called ‘How to build a brain’ which will describe in depth what made the astonishing results possible.
But what are the results?
Well that looks like number recognition. In fact that’s what it is. SPAUN – that’s how the scientists refer to their frankenstein-brain – is capable of solving 8 different tasks now. One of them is number recognition. There are videos of all 8 tasks being performed.
The Semantic Pointers are named after the pointers usually common in computer science:
“Higher-level cognitive functions in biological systems are made possible by semantic pointers. Semantic pointers are neural representations that carry partial semantic content and are composable into the representational structures necessary to support complex cognition.
The term ‘semantic pointer’ was chosen because the representations in the architecture are like ‘pointers’ in computer science (insofar as they can be ‘dereferenced’ to access large amounts of information which they do not directly carry). However, they are ‘semantic’ (unlike pointers in computer science) because these representations capture relations in a semantic vector space in virtue of their distances to one another, as typically envisaged by connectionists. “
Isn’t technology great when it brings families closer together, even when they are thousand miles apart?
Home automation does not only mean that you are going to flip some switches and sensor away in every imaginable way. It also means creativity. And being creative with the functionalities at hand is really what makes home automation so interesting.
It’s those creative ways that adds use to the nerdy home automation switches and sensors. It’s what adds practicality.
Good Night Lamp is such a creative solution that makes use of home automation hardware and the internet. To understand the concept, watch a video:
“The Good Night Lamp is a family of connected lamps that lets you communicate the act of coming back home to your loved ones, remotely.”
Well I don’t know if it really needs specialized hardware like those Good Night Lamp products. But certainly if you have some sensory and the ability to flip switches it is fairly easy to come up with workflows and things that should happen when the circumstances are right. In fact I do not believe in highly specialized products like a single-purpose lamp. But I do believe, if those lamps are connected to a network and if you can access them through some sort of API, that those types of products will pave the way to a connected world we only know from science fiction yet.
Another good solution to this is the long promised IP capable light bulb. Engineers were using the “light bulb with an ip adress” as an example for IPv6 for years now. And it seems that the time has come when we really want to assign an ip adress to every lightbulb in our home.
LIFX is a good start concept and in a couple of months there will be more manufacturers who are offering networked light bulb solutions.
I was asked if it would be possible to get the ELV MAX! Cube interfacing functionality outside of h.a.c.s. – maybe as a library. Sure! That is possible. And to speed up things I give you the ELV MAX! Cube C# Library called: MAXSharp
It’s a plain and simple library without much dependencies – in fact there’s only some threading and the FastSerializer. Since I am using this library with h.a.c.s. as well I did not remove the serializer implementation.
There’s a small demo program included which is called MAXSharpExample. The library itself contains the abstractions necessary to get information from the ELV MAX! Cube. It does not contain functionality to control the cube – if you want to add, feel free it’s all open sourced and I would love to see pull requests!
The architecture is based upon polling – I know events would make a cleaner view but for various reasons I am using queues in h.a.c.s. and therefore MAXSharp does as well. The example application spins up the ELV MAX interfacing / handling thread and as soon as you’re connected you can access all house related information and get diff-events from the cube.
Any comment is appreciated!
Source 1: State of Reverse Engineering
Source 2: https://github.com/bietiekay/MAXSharp
Workflows are important – and having a lot of switching possiblities and even more sensors that measure things it begins to become important to be able to implement workflows behind all that hardware.
It’s nice to be able to switch light on and of when you want to. But isn’t it even better to have some sort of workflow behind all sorts of triggers. Think of the possibilities!
If this then that is a service to help you define very simple workflows:
Want an example?
It knows a lot of ‘this’ and a lot of ‘that’. So give it a try or even better, add your own home automation software as ‘this’ and ‘that’ 🙂
Source 1: https://ifttt.com
A couple of days ago the well known and much read Nerdcore weblog author created a page he calls NC-Sources which lists all the sources he has in his RSS reader to get new information from. As you can imagine, this is pure gold for those who want to get interesting links to all-nerd pages.
Unfortunately NC-Sources is just available as a web-page which lists the name and the RSS feed URL. You cannot import that into your RSS Reader to use it for your own informational needs.
Here I am to the rescue. I’ve taken all the URLs from that NC Source page. That resulted in a file that lists the page url and the rss-feed url in alternating lines. A short trip to the command line and the use of awk helped to filter just the rss-feed urls to a new file and that was filled into an opml generator.
So now you can download the OPML file to import it into your own RSS reader. Get it here.
Those who know me well know that I am a strong believer of artificial intelligence. We’re not there yet. Not even close, I don’t know if we (as in humanity) even left the launchpad. But I strongly believe that it will be possible to simulate human thoughts – maybe not in the way AI is defined:
“The field was founded on the claim that a central property of humans, intelligence—the sapience of Homo sapiens—can be so precisely described that it can be simulated by a machine.This raises philosophical issues about the nature of the mind and the ethics of creating artificial beings, issues which have been addressed by myth, fiction and philosophy since antiquity.Artificial intelligence has been the subject of optimism,but has also suffered setbacksand, today, has become an essential part of the technology industry, providing the heavy lifting for many of the most difficult problems in computer science.” (Wikipedia)
More on that in another article in the future since I started working on that subject earlier and now I come across a lot of authors and mostly science fiction books that deal with that topic.
Now there is a new book by Ray Kurzweil. It’s called “How To Create A Mind” and deals with the topic of how human thoughts come to be and how the human mind seems to work.
“Now, in his much-anticipated How to Create a Mind, he takes this exploration to the next step: reverse-engineering the brain to understand precisely how it works, then applying that knowledge to create vastly intelligent machines.
Drawing on the most recent neuroscience research, his own research and inventions in artificial intelligence, and compelling thought experiments, he describes his new theory of how the neocortex (the thinking part of the brain) works: as a self-organizing hierarchical system of pattern recognizers. Kurzweil shows how these insights will enable us to greatly extend the powers of our own mind and provides a roadmap for the creation of superintelligence—humankind’s most exciting next venture. We are now at the dawn of an era of radical possibilities in which merging with our technology will enable us to effectively address the world’s grand challenges.”
Source 1: http://howtocreateamind.com/
Source 2: http://en.wikipedia.org/wiki/Artificial_intelligence
The first signs of the upcoming camera board for the raspberry pi are showing. During the Electronica 2012 fair RS showed the board to the public for the first time.
Since it’s going to be a 25 Euro add-on for the Pi the specification is quite impressive. The OmniVision OV5647 is used as the Image Sensor – it’s bigger brother is used in iPhone 4. OmniVision says:
“The OV5647 is OmniVision’s first 5-megapixel CMOS image sensor built on proprietary 1.4-micron OmniBSI™ backside illumination pixel architecture. OmniBSI enables the OV5647 to deliver 5-megapixel photography and high frame rate 720p/60 high-definition (HD) video capture in an industry standard camera module size of 8.5 x 8.5 x ≤5 mm, making it an ideal solution for the main stream mobile phone market.
The superior pixel performance of the OV5647 enables 720p and 1080p HD video at 30 fps with complete user control over formatting and output data transfer. Additionally, the 720p/60 HD video is captured in full field of view (FOV) with 2 x 2 binning to double the sensitivity and improve SNR. The post binning re-sampling filter helps minimize spatial and aliasing artifacts to provide superior image quality.
OmniBSI technology offers significant performance benefits over front-side illumination technology, such as increased sensitivity per unit area, improved quantum efficiency, reduced crosstalk and photo response non-uniformity, which all contribute to significant improvements in image quality and color reproduction. Additionally, OmniVision CMOS image sensors use proprietary sensor technology to improve image quality by reducing or eliminating common lighting/electrical sources of image contamination, such as fixed pattern noise and smearing to produce a clean, fully stable color image.
The low power OV5647 supports a digital video parallel port or high-speed two-lane MIPI interface, and provides full frame, windowed or binned 10-bit images in RAW RGB format. It offers all required automatic image control functions, including automatic exposure control, automatic white balance, automatic band filter, automatic 50/60 Hz luminance detection, and automatic black level calibration.”
That sensor delivers RAW RGB Imagery to the RaspberryPi through the onboard camera connector interface:
And the part that impressed me the most is that that 5 Megapixel sensor delivers it’s raw data stream and it gets h264 compressed directly within the GPU of the Raspberry Pi. 30 frames per second 1080p without noticeable CPU load – how does that sound? – Not bad for a 50 Euro setup!
In times when mobile phone cameras produce pictures of 2 MBytes each and decent DSLR cameras produce pictures in the range of more than 20 Mbytes each – not speaking of the various sensors around the house the question of how all of this is going to be stored is an interesting one.
Prices for mass storage is dropping for years and sized of hard disks are getting bigger and bigger. 3 Tbyte drives are fairly cheap now. Cheap enough to consider serious redundancy even for home use.
Having that home automation hobby and having very specific needs when it comes to home entertainment or even watching TV (we don’t watch live-tv…) we have a relatively huge demand for storage space. That way we are already storing over 10 Tbyte of data, fully encrypted, redundant and backed-up.
Our file server infrastructure grew with the needs over the years.
It started way back in 2003 when I set-up the first fileserver for my apartment back then. It was a fairly huge 19 inch case with 5 hard disks (100 Gbyte each). This machine was filled in 2005 and needed replacement.
We’re in IDE land back then. Because the system hardware died on me due to a power surge all the disks and a new mainboard were seated in a new case with room for a lot of disks.
One interesting detail might be that I consistently used Windows Server for that purpose.
The machine always wasn’t just a fileserver. It was smtp, imap, nntp and media server all the time. That lead to a growing demand of CPU and memory resources. It started with an 800 Mhz AMD Athlon (which died quickly) and for the next years to come I used a 2.8 Ghz Intel Pentium 4. Everything started with Windows Server 2003 – bought in the Microsoft Store when I was a Microsoft employee.
Diskspace demand kept growing and in 2009 a new case, new mainboard + memory and new disks where due.
Since 2009 a Core4Quad Q9550 with 2.8 Ghz and 16 Gbyte of Memory is the heart of our fileserver. Since we’re frequently live-transcoding video streams to feed iPads and iPhones around the house that machine has plenty of grunt to feed the demand. We can have 2 iPhones and 2 iPads playing 720p content without getting stutters. Back in 2009 we also switched to a mixed IDE and SATA setup as you can see in the picture:
Plenty of room when the new case arrived – it was getting crowded just 2 years later in 2011. Every seat was taken – which means 13 disks are in that case and 1 attached through USB.
That adds up to more than 16 Tbyte of raw storage. In 2011 we also upgraded to Windows Server 2008. We never lost a bit with that operating system, not under the heaviest load and even through serious hardware malfunctions. A lot of disks of those 13 died throughout the years: Almost 1 every 2 months was replaced – most of them through extended waranties – of course we have a spare always ready to take the place. Only one time I had to rush to a store to get a replacement drive when two disks failed short after each other. That’s why there’s that 2 Tbyte drive in the 1.5 Tbyte compound…
So it’s getting full again. Since that case isn’t really holding more disks and replacing them is getting harder because of the tight fit the idea was born to now add a bigger case but to just add a NAS/SAN which holds between 6 to 8 disks at once, comes with it’s own redundancy management and exports one big iSCSI volume.
That said a network card was added to the fileserver and a QNAP TS-859 Pro+ 8-bay appliance was bought. This one is a shiny black device which uses less power then an aditional case with extra cpu and memory would have use and after calculating through a number of combinations it’s even the cheapest solution for an 8 drive set-up.
After some intensive testing it seems that the iSCSI approach is the most robust one. Since I am just done with testing the appliance the next step is to buy drives. So stay tuned!
Back in 2006 I wrote about a new technology which the also new company Geomerics was demoeing.
Back in 2006 everything was just a demo. Now it seems that Geomerics found some very well known customers and without noticing a lot of the current generation games graphics beauty comes from the capabilities real time radiosity lighting is adding to the graphics.
“Geomerics delivers cutting-edge graphics technology to customers in the games and entertainment industries. Geomerics’ Enlighten technology is behind the lighting in best-selling titles including Battlefield 3, Need for Speed: The Run, Eve Online and Quantum Conundrum. Enlighten has been licensed by many of the top developers in the industry, including EA DICE, EA Bioware, THQ, Take 2 and Square Enix.” (Source)
There even is a more updated version of the demo video:
With todays processing power and the faults of current generation digital video cameras you can have a lot of fun – if you know how:
The above demonstrated effect is called Time Remapping. The description of the video tells us more about the effect itself:
The effect was discovered accidentally by a photographer called Jacques Henri Lartigues at the beginning of the 20th century (in 1912 to be precise). He took a picture of a race car with eliptical deformed tires – an effect caused by the characteristics of the camera he was using.
There you are – you’ve spent hundreds of hours, maybe together with friends, in a game called Minecraft. You mined and you crafted. And you built yourself your own world. Out of blocks.
“Minecraft is a game about breaking and placing blocks. At first, people built structures to protect against nocturnal monsters, but as the game grew players worked together to create wonderful, imaginative things.
It can also be about adventuring with friends or watching the sun rise over a blocky ocean. It’s pretty. Brave players battle terrible things in The Nether, which is more scary than pretty. You can also visit a land of mushrooms if it sounds more like your cup of tea.”
Those who haven’t played Minecraft yet – you’re missing out a lot. It’s fun and addictive. It seems pretty dull when you don’t know it. As soon as you got immersed in it you immediately see that it’s a lot bigger and the possibilities are a lot more varying than at first sight.
With all those blocks you can basically build your own world and humongously huge objects. It obviously takes a while in most cases because you (until you start using tools and mods) need to fit each block to the other in order to create those big objects.
So imagine you got your own world and you want to create nice renderings of it to hang on your real-world-appartment walls? You can use a very simple to use and thankfully free (open sourced) tool to do that.
It’s called McObj and it uses blender to render the exported geometry. Get it and send your renderings!
Source 1: https://github.com/quag/mcobj
Source 2: http://quag.imgur.com/minecraft__blender
Source 3: https://minecraft.net/
Source 4: http://www.pcgamer.com/2012/11/29/minecraft-renders-azeroth-and-the-pc-gamer-server/
Progress is showing in regards of the next incarnation of the famous Hypertext transport protocol aka http. Despite the fact that those 4 letters got banned from modern browser adress bars it’s still the cornerstone of everything your browser does with the network.
Based upon the work of Google and their SPDY implementation it comes with a lot of things that come in handy when thinking about modern demands for security, performance and multi-channel-data-transport.
In November 1998 there was a book released about file system design taking the Be File System as the central example.
“This is the new guide to the design and implementation of file systems in general, and the Be File System (BFS) in particular. This book covers all topics related to file systems, going into considerable depth where traditional operating systems books often stop. Advanced topics are covered in detail such as journaling, attributes, indexing and query processing. Built from scratch as a modern 64 bit, journaled file system, BFS is the primary file system for the Be Operating System (BeOS), which was designed for high performance multimedia applications.
You do not have to be a kernel architect or file system engineer to use Practical File System Design. Neither do you have to be a BeOS developer or user. Only basic knowledge of C is required. If you have ever wondered about how file systems work, how to implement one, or want to learn more about the Be File System, this book is all you will need.”
If you’re interested in the matter I definitely recommend reading it – it’s available for free in PDF format and will help to understand what those file system patterns are all about – even in terms of things we still haven’t gotten from our ‘modern filesystems’ today.
Source 1: http://www.nobius.org/~dbg/
It’s been some weeks since I wrote a status update on the ELV MAX! cube protocol reverse engineering and integration into my own home automation project called h.a.c.s..
So first of all I want to give a short overview over what has been achieved so far:
- I wrote a C# library, highly influenced by a PHP implementation from the domotica forum, which allows you to continuesly get status information from the ELV MAX! cube with current (1.3.6) firmware. It is tested so far with a fairly big set-up for the ELV MAX! cube (see below)
- I was able to integrate that library into my own home automation project called h.a.c.s. – There the ELV MAX! cube is just another device, alongside a EzControl XS1 and a SolarLog 500. The cube is monitored using my library and diff-sets as well as status information are stored automatically with the h.a.c.s. built-in mechanisms. In fact you can access for example the window shutter contact information just like you would with any other door contact in the EzControl XS1.
- You can use events coming from the ELV MAX! cube to create new events – how about switching off/on devices when opening/closing windows?
- Every bit of information from all integrated sensor monitoring and actor handling devices come together in h.a.c.s.
I started the reverse engineering with just one shutter contact and one thermostat. After all my test were successful I went for the big package and ordered some more sensors. This is how the setup is currently configured:
I’ve learned a lot of interesting things about the ELV MAX! cube hardware and software. One is that you need to be ready for surprised. The documentation of the cube tells you the following:
Did you spot the funny fact? 50 devices – we’re well below that limit. 10 rooms – holy big mansion batman! We’re well over that. How is that possible? Well take it as a fact – you can create more than 10 rooms. And that is very handy. I’ve created 13 rooms and there are probably more to come because those shutter contacts are quite cheap and can be used for various other home automation sensory games. The tool to set-up and pair those sensors just came up with a notice that said “Oh well, you want to create more than 10 rooms? If you’re sure that you want that we allow you to, but hey, don’t blame us!”. Cool move ELV! – As of now I haven’t found any downside of having more than 10 rooms.
All my efforts started with firmware version 1.3.5. This firmware seemed to have some severe memory leaks – because just by retrieving the current configuration information every 10 seconds the device would stop communicating after more then 48 hours. Only a reboot could revive it – sometimes amnesia set in which led to a house roundtrip for me.
With some changes in the library (like keeping the connection open as long as possible) and a new firmware version 1.3.6. the cube was way more cooperative and hasn’t crashed for about 1 month now (with 10 seconds update times).
So what does my library do? It is designed to run in it’s own thread. When it’s started it opens a connection to the cube and retrieves the current status and configuration information. Those informations are stored in an object called “House”. This house consists of multiple rooms – and those rooms are filled with window shutter contacts and thermostats. All information related to those different intances are stored along with them. The integration into h.a.c.s. allows the library to generate sensor and actor events (like when a temperature changes, a window opens/closes) which are passed back to h.a.c.s. and handled in the big event loop there.
With all that ELV MAX! cube data I wanted to plug a quite nice tool that I am using in the iPhone and the iPad. It’s called “Moni4home” and it allows you to control the EzControl XS1 directly. Because it’s only accessing the EzControl XS1 I used h.a.c.s. to “inject” additional sensor data into the standard EzControl XS1 data. So basically data flow is like this: iPad app accesses h.a.c.s. which acts as a proxy. h.a.c.s. retrieves the EzControl XS1 sensor and actor data and injects additional virtual sensors like those from the ELV MAX! cube. h.a.c.s. then sends that beefed up data to the iPad app. Voilá!
After the successful integration of the ELV MAX! cube I’ve started to work on the next bit of networking home automation equipment in my house – a solar panel data logger called “Solar-Log 500”. This device monitors two solar power inverters and stores the sensory data.
“Funny” story first: this device has the same problem like the ELV MAX! cube. When you start to poll it every 10 seconds (or less) it just stops operating after about 20 hours. Bear in mind: In case of the Solar-Log I just http-get a page that looks like this in the browser:
And by doing so every 10 seconds the device stops working. I am using the current firmware – so one workaround for that issue is to planable reboot the Solar-Log at a time when there is no sun and therefore nothing to log or monitor.
Beside that it’s a fairly easy process: Get that information, log it. Done.
So there you have it – h.a.c.s. interfacing with three different devices and roughly 100 sensors and actors over 434mhz/868mhz, wireless and wired network. There’s still more to come!
A lot of people seem to dive into home automation these days. Apparently Andreas is also at the point of starting his own home automation project. Good to know that he also is using the EzControl XS1 and in the future maybe even the ELV MAX! cube. Party on Andreas!
Source 1: ELV MAX! cube progress
Source 2: Reverse Engineering the ELV MAX! cube protocol
Source 3: ELV MAX! cube – home automation for the heating
Source 4: http://www.solar-log.com/de/produkte-loesungen/solar-log-500/uebersicht.html
Source 5: h.a.c.s. sourcecode
Source 6: http://monitor4home.com/Beschreibung.html
Source 7: http://www.aheil.de/2012/11/06/hack-the-planet-architectural-draft/
Some weeks ago I heard about a new audio codec which is being developed as open source – very similar to vorbis – the previous open source approach to audio codecs.
This time it seems that they’ve got some standardization into the play so it might be more successful than vorbis was.
“Opus is a totally open, royalty-free, highly versatile audio codec. Opus is unmatched for interactive speech and music transmission over the Internet, but also intended for storage and streaming applications. It is standardized by the Internet Engineering Task Force (IETF) as RFC 6716 which incorporated technology from Skype’s SILK codec and Xiph.Org’s CELT codec.”