- Family and Friends
Archive for category N3RD
It’s a common use case: you’ve got some JSON formatted data and you want to interface with it using your favourite programming language C#. You can write the appropriate classes yourself, or you could use the fabulous json2csharp helper page.
OpenType is a font format which I personally might have underestimated in the past. Well you know – fonts and stuff. This all seemed not too interesting up until now. Now that changed dramatically when a font came to my attention which can be used for various purposes and as a font does not resemble the normal numbers and characters scheme. But what can it be used then if not to type numbers and characters?
Well. What about typing graphs?
Everything in the above image is generated by a font… like in your Word-processor (if it uses that font)
“Designed by Travis Kochel, FF Chartwell is a fantastic typeface for creating simple graphs. Driven by the frustration of creating graphs within design applications (primarily Adobe Creative Suite) and inspired by typefaces such as FF Beowolf and FF PicLig, Travis saw an opportunity to take advantage of OpenType technology to simplify the process.
Using OpenType features, simple strings of numbers are automatically transformed into charts. The visualized data remains editable, allowing for hassle-free updates and styling.”
Source 1: https://www.fontfont.com/how-to-use-ff-chartwell
Do you know what happens during the push of the power button and typing your log-in information inside of your computer? No? You should. At least from a software side. Not that it is necessary to use a computer. But in order to understand what this wonderful machine does and why.
For those teaching and learning purposes the Raspberry Pi is a perfect device. It’s cheap and now there is a course you can take online which shows you – starting from the very beginning – how to get the device up and running and how to make it do what you like. And that’s without installing an operating system. You are about to write your very own.
“This website is here to guide you through the process of developing very basic operating systems on the Raspberry Pi! This website is aimed at people aged 16 and upwards, although younger readers may still find some of it accessible, particularly with assistance. More lessons may be added to this course in time.”
It certainly is just me thinking: this home automation / smart home thing gains more momentum every week. Now there’s a java based home automation bus initative taking care of the software standardization side. Quite interesting. And beside all that they had some fantastic ideas how a user interface for those things should look like. Like for example how you would interact with your house while planning when things power on and off. Use Google Calendar! This is just plain genius!
“The open Home Automation Bus (openHAB) project aims at providing a universal integration platform for all things around home automation. It is a pure Java solution, fully based on OSGi. The Equinox OSGi runtime and Jetty as a web server build the core foundation of the runtime.
It is designed to be absolutely vendor-neutral as well as hardware/protocol-agnostic. openHAB brings together different bus systems, hardware devices and interface protocols by dedicated bindings. These bindings send and receive commands and status updates on the openHAB event bus. This concept allows designing user interfaces with a unique look&feel, but with the possibility to operate devices based on a big number of different technologies. Besides the user interfaces, it also brings the power of automation logics across different system boundaries.”
I especially like the idea of that calendar integration – sending scripts through an appointment is a great idea – having some sort of scripting language is another one. A little bit on the marketing side is the option to chat with your house through XMPP / Jabber… that might take the idea a little bit too far out – but who would want to blame them? Fantastic stuff!
Source 1: http://www.openhab.org/
Source 2: http://kaikreuzer.blogspot.de/2012/08/openhab-1.html
Everyone is excited about the Mars Science Laboratory cruising on Mars. NASA/JPL pulled an unbelievable stunt to get that almost-a-ton Rover on that planet. Well done!
Alongside with the landing some source reported that NASA/JPL now has started to provider source code access to some of it’s internal projects. And beside being right this is not the first time they’ve provided source code access. They got a whole page with all their projects open sourced. Some of those are quite huge projects – big like those you would fly to Mars with.
Even more interesting from a historical perspective are the older open sourced projects. Like the complete manuals and source code listings of the gemini and apollo projects.
It’s a great read overall and it let’s you wondering with how little you can achieve great goals.
For several years now I am interested in this home automation thing – I even got a little bit of my own home automation going. But with websites like domotica you can get an idea of what is achieveable and how it might look for the people actually using it every day.
I had a couple of hours to tinker with my ELV MAX! Cube and there is some progress with the protocol reverse engineering.
Of course there is the domotica forum helping out with some information the guys over there have found but in addition to their very helpful findings I want it to be integrated into h.a.c.s. – and along with it I maybe want to have a way to find eventual protocol changes quick and easy in the future.
So yesterday I partied on the ‘first contact’ – today I am a bit deeper into the protocol itself:
Here are some explanations to the picture:
When a tcp connection to the cube is opened you can immediately read from it – the cube is throwing information at you. There’s always a character at the beginning of each line which marks the type and beginning of the message.
There seem to be these types of messages in the first package of information:
- H – Header maybe?
- it contains the serial number of your cube, the RF address, the firmware version and several other things like time information
- M – Metadata?
- this seems to be some kind of global metadata list, containing the rooms with their IDs (it’s the %) in the screenshot). Furthermore it contains the serial numbers and names of the devices in that room – at the moment there’s just a window-state-sensor in that first room called “Fensterkontakt 1”
- C – Configuration?
- since there are multiple C messages these seem to contain detailed configuration data specific to a device in the MAX! network. Each device seems to be addressed by a RF Address and it’s serial number.
- the first C message in the screenshot is associated to the cube itself
- the second C message is associated to the window-state-sensor – you can clearly see in there the room id “%)” and the serial of the window-state-sensor.
- L – live status?
- this message seems to contain room status information. In our case there is only the room with id “%)”. When the window-state-sensor changes state the last byte changes value – interesting, eh?
On the coding side I’ve got several things set-up in my little debug tool. I’ve wrapped those message types into various classes to handle them more easily later on in h.a.c.s.. Furthermore I used a little decompiler-wisdom to extract some more information from the included ELV MAX! cube software.
Thanks to german UrhG paragraph § 69e (german copyright law) I am allowed to decompile the included software in order to achieve interoperability (and only that). That’s exactly what I would like to achieve: Interoperability. And for the record: besides that I also filed a support request to ELV in which I ask them if I could get access to a presumably existing documentation of that protocol.
While waiting on that documentation I am using JD-GUI as a decompiler user interface for java – since the software of the cube is written in java.
There are many interesting things in there but it’s a slow process to get ahold of all the things necessary. There are already some very nice things showing up. Like when you want to know if there’s a cube (or more) in the network you just need to send a multicast ip packet containing a characteristic signature and all the cubes in your network will try to connect back to you with some basic information – nice, isn’t it? Or what about that AES Encryption/Decryption that seems to be built into the cube? Yes that’s right! It seems to be possible to send commands to either encrypt or decrypt according to the AES. Thoughtfully these commands are marked with ‘e’ and ‘d’. Or that if you send “l:” as a command with CR+LF at the end you get a device listing with all stats… and so on.
Some open question to EQ-3/ELV for the end of this article:
- Why this strange protocol? Why all the work on both sides? Just because an HTTP server implementation with a RESTful service would have been that more difficult?
- Base64 encoded data? The 90s called, they want their 8th bit back.
- why that complex local webserver approach when you could have done everything in a java app anyways?
That’s it for today, I just pushed a feature to the Git repository which allows you to run whatever command you like on your cube with the debugging tool:
Source 1: http://www.domoticaforum.eu/viewtopic.php?f=66&t=6654&sid=f8f912914163cb44d447cfa3de44d63d
Source 2: http://en.wikipedia.org/wiki/Decompiler
Source 3: http://www.gesetze-im-internet.de/urhg/__69e.html
Source 4: http://java.decompiler.free.fr/?q=jdgui
Source 5: http://en.wikipedia.org/wiki/Advanced_Encryption_Standard
Ever wondered how a software finds out that this file named “filename” is a pdf, jpeg, movie? There are several thousands, probably hundreds-of-thousands of fileformats out there. Some of them are used many times a day without us even noticing. We’re just moving an image from A to B not caring about what constitutes an image file and what makes a jpeg different to a png image.
Now for pure academic reasons there is one file that is many (no, not borg). It’s a file that is:
“CorkaMIX.exe is simultaneously a valid: * Windows Portable Executable binary * Adobe Reader PDF document * Oracle Java JAR (a CLASS inside a ZIP)/Python script * HTML page
It serves no purpose, except proving that files format not starting at offset 0 are a bad idea. Many files (known as polyglot) already combines various langages in one file, however it’s most of the time at source level, not binary level.”
For several years now I am building my own home automation tools by putting together existing hardware and self-written software. As the central software core of my home automation system I use h.a.c.s. – “home automation control server” which I put up as open source software on GitHub.
Throughout the years I was able to embedd a lot of daily tasks and measurements in one place which can be accessed by a simple web page. It currently looks like this:
You can find some articles on this blog about h.a.c.s. if you want to know more about it.
As of today I can control and measure the states of switches, windows, doors, temperature and humidity and power consumption. Scenarios like “when this door opens, switch on that light” are easy things to do with h.a.c.s.
Now “Winter’s coming!”. And therefore I want to take control of the heating of each and every room in the house. I want to set a goal for a temperature and I want the heating to fire up or cool down with that goal. And of course I want to monitor manual changes of each and every radiator in the house.
Last week then I stumbled upon a piece of kit called “ELV MAX! Cube”. It’s a white cube (as the name implies) which offers a USB port from which it is powered and an RJ-45 ethernet port which connects the cube to the home network.
The cube itself does not draw much power and it can be powered by the routers USB port easily. It allows you to connect some peripherals using 868 mhz rf. Those peripherals can be: window state sensors (closed/open) and thermostats to control the radiators (and a switch but, well… hopefully not necessary).
It comes with it’s own user interface – a java application that connects to the device and allows you to configure it. Quite nice – it runs on Windows and Mac. You can use a cloud service to control the device over the internet, but I have no intention in trying that out right now.
My plan is to extend h.a.c.s. to get information from the cube and handle them and in the end even control the cube by setting temperatures and controlling the outcome of those changes.
As of now there are some efforts to decode the quite interesting protocol the cube is talking. You communicate with the cube over TCP (my cube listens on port 62910).
Currently I am building a small debug application which allows me to experiment with the output of the cube faster than plain telnet would. And within this I had the first contact tonight:
As always all my efforts can be seen in the hacs repository.
Source 1: https://github.com/bietiekay/hacs
Source 2: http://www.schrankmonster.de/?s=hacs
Source 3: http://www.elv.de/max-cube-lan-gateway.html
Source 4: http://www.domoticaforum.eu/viewtopic.php?f=66&t=6654
It appeared to me that I stopped working with a decent keyboard since I moved completely to Macs at home. I was using the keyboards the machine came with and not always does Apple deliver the best possible keyboard for the money.
So I tried to turn back to my trusty IBM PS/2 Model M last week and I had to find out that somehow the actively powered USB to PS2 adapter I had is got lost. A passive one just doesn’t cut it and the keyboard does not work at all.
I remembered that in 2006 I wrote about a back-then-new keyboard that resembled the fantastic Model M. Voilá! They even worked on their keyboards since 2006 and improved them 🙂
A little bit more than 6 years after writing first about the product I got me a “das Keyboard Ultimate S EU”.
First verdict: It is awesome!
It’s expensive, that’s true. But if just feels right typing on it. I can see me writing a lot of stuff for longer periods on that keyboard 🙂
It’s nice to have business partners who sponsor racing teams. Even nicer is when they invite you over to enjoy a couple of porsches racing the only german city race course called Norisring.
I always wanted to watch a race there live. And it was a total blast. As guests of Hermes and Hermes Attempto Racing we had seats on the Porsche bleachers and we had access to the Porsche Hospitality center. The weather was great, maybe even a little bit too good – over 35 C / 95 F with a merciless sun so that even the tarmac melted during a race which lead to the race being cancelled.
Just a couple of days ago – after a waiting time of more than half a year – my personal raspberry pi board arrived. Fantastic!
It’s small. Oh yes, it’s very very small.
What is the Raspberry Pi you may ask:
“The Raspberry Pi is a credit-card sized computer that plugs into your TV and a keyboard. It’s a capable little PC which can be used for many of the things that your desktop PC does, like spreadsheets, word-processing and games. It also plays high-definition video. We want to see it being used by kids all over the world to learn programming.”
For under 40 Euro you get a huge choice of I/O interfaces like USB, Ethernet, HDMI, Audio and Multi Purpose IO pins you can play with if you’re into hardware hacking. This small card is running a fully blown linux and because it has a dedicated graphics core which can hardware decode and encode 1080p h264 it’s definitely a good choice for a home mediacenter (yes, XBMC runs on it.)
It draws so little power that you could use solar panels to power it. It’s all open and sourced and I will use it for a couple of things in the household. Like a cheap Airplay node. Or a more intelligent sensor node for home automation. This thing seriously rocks – finally a device to play with – with reasonable horse-power.
There are many things which are underestimated when team leads think about their team and possible actions to drive progress.
One of those things is that a team needs information to maintain and gain velocity. You cannot expect everyone to know just out of the blue what is important and in which direction everything is moving. To let everyone know and to develop that direction it’s important to share information as much as possible. It’s important to give everyone access to the information necessary to make a better job.
That’s why we had a build monitor at sones. We had a tool that displayed the current status of our build servers to all developers. Everytime someone committed a change, those build servers got this commit, built it and tested it with automated tests. The status of that could be seen by all developers as things happened.
So within seconds everyone could see if his commit did break something. Even better: Everyone could see. Everyone cared that the build needed to be working, that tests needed to pass. It was everyones job to do the housekeeping. When we switched from Team Foundation Server to GIT and Jenkins this status display needed to be replaced – you could immediately tell that things went from good to not-so-good in terms of build stability and automated testing.
Today I had the opportunity to take a tour of the Thomann logistics center. Standing in the support department I had this in front of me:
There were like 6 big status screens displaying incoming call status of the day, sales figures and other statistics important to those who work there. It’s a very important and integrated way to keep information flowing.
Since I am with Rakuten I thought about having a new status board set-up for my team. Something that might be inspired by the awesome status board which panic has built:
Since in addition to sones there are a lot of more things to track and handle (code, deployment, operations, overall numbers) I think such a status board will be of invaluable worth for the team.
You know that: You have just stumbled upon a great and informative YouTube channel. It’s full of videos you would like to watch but to do that you need to have internet access in any case. And of course that internet access needs to be as fast as possible to cope with the video quality you would like to watch.
If only it would be possible to download a video from YouTube, store it locally and watch it whenever you got the time. Maybe you want to take that video with you on that great, internetless self-awareness trip…
Now there are a lot of tools that allow you to download YouTube clips manually. I used BYTubeD for that purpose. It is a nice and easy to use Firefox Add-On which can be started whenever a YouTube video appears in any page.
After you’ve started into BYTubeD you can select which of the videos on the page you would like to download and what quality you would like to get.
All this works very well if you only want to download something once every while. Problems come up if you want to download regular postings…
I’ve subscribed to several – to me – very interesting YouTube channels. These get updated almost every day. The only option for me to keep track with them is to take the time, surf YouTube and use BYTubeD to download manually if there is anything new. Now this was a waste of time for me so I automated it.
I wrote a small tool I call “YouTubeFeast” – because it allows you to feast on YouTube… yeah I know. Now this tool is designed to run on a linux or windows machine in the background and scan in configurable intervals for new videos. If it finds new videos it downloads them in the quality you pre-configured to a folder you configured. It couldn’t be easier.
It’s open-source (GPLv2) and I’ve made it publicly available on GitHub. You can even find a pre-compiled binary version there which is ready-to-run.
The configuration file “YouTubeFeast.configuration” is a plain and simple text file. Use your favourite text editor and obey some simple rules:
- any line beginning with # is a comment
- any line not beginning with a # is a download-job
- any download job consists of the following, tabulator separated parameters:
- the URL of the video page / channel homepage / overview
- the desired quality (360p, 720p, 1080p)
- the path to store the videos
- the interval (in hours) to check for new stuff
- don’t forget: tabulator separates parameters (take a look into the example configuration file…)
After configuring the only thing you need to do is to start YouTubeFeast. It will then go through all the jobs and download video files – as soon as it comes across an already downloaded file it stops that specific job.
That’s all about it. If you got any comment or suggestions for improvement please let me know.
Yesterday @simcup wrote on twitter about that he is currently downloading the whole Jamendo catalog of Creative Commons music.
Although I already knew Jamendo it never occurred to be to download their whole catalog. Since I am a fan of choice I immediately thought about how I could download the catalog too. Since the only clue was a cryptic uri-like text how to achieve that it suddenly sounded like a great idea to write a universal tool and release it as open-source. This tool should allow users to download the whole catalog and keep their local jamendo mirror in sync with the server. So anytime new artists, albums or tracks are added the user does not need to download them all again.
So the only thing I had as a starting point was that cryptic uri pointing me to something I’ve never heard of called Rythmbox. Turns out that this is a GNOME music player application which has Jamendo integration. After some clueless poking around I decided to take a look at the source of Rythmbox, especially the Jamendo module.
This module is written in python and quite clean to read. And just by looking at the first lines I came across the interesting fact that there is a almost daily updated XML dump of the Jamendo catalog available from Jamendo. Hurray! Since Jamendo wants developers to interact with the platform they decided to put a documentation online which allows anyone to write tools and stream and download tracks. After all the clues I found I finally ended up on this page.
So there are the catalog download, track stream and torrent uris necessary to download the catalog. Now the only thing that is needed is a tool which parses the XML and creates a nice folder structure for us.
Parsing XML in C# (my prefered programming language) is easy. Basically you can use a tool called XSD.exe and let it generate first the XSD from the XML and then ready-to-use C# classes from that XSD.
After doing all that actually reading the whole catalog into a useable form breaks down to just three lines of code:
Isn’t it great how modern frameworks take away the complexity of such tasks. At this point I’ve already parsed the whole catalog into my tool and only wrote three lines of code. The rest was generated automatically for me. The best of all – this also works on non-windows operating systems when you use mono.
When the XML data is parsed and available in a nice data structure it’s easy to iterate through all artists, all albums and all tracks and then download the actual mp3 or ogg. And that’s basically what my tool does. It takes the XML, parses it, and downloads. It will check before downloading if the track already exists and will only download those added since the last run.
Additionally since I am deeply involved into the development of the GraphDB graph database at sones I want to make use of the Jamendo data and the graph structure it poses. Since the directory structure my tool is generating is only one aspect how you could possibly look at the data it’s quite interesting to demonstrate the capabilities of GraphDB based on that data.
The idea behind the graph representation of the data is that you could start from almost any starting point imaginable. No matter if you you start from a single track and drill up into genre and artists, or if you start at a location and drill down to tracks.
So what the Downloader does in matters of GraphDB integration is that it outputs a GraphQL script which can be imported into an instance of GraphDB.
The sourcecode of my tool is available on github and released unter the BSD license – feel free to play with it and to contribute.
Source 1: http://www.jamendo.com
Source 2: https://github.com/bietiekay/JAMENDOwnloader
Two weeks ago I had read an article about a “replacement for papernotes” product called “Boogie Board”. The company behind the product claims to replace paper with the bold slogan of “say goodby to paper”.
Well what is it? Basically it’s a liquid crystal display without the logic to adress specific pixels. So think of it like taking the liquid crystal part and leaving out all the transistors and logic to actually display something. Then add a pen or even your finger nail and you can “write” on that display – what’s happening is that obviously the crystals get pushed aside and the background of the “display” shines through – this background is white so when you write on the boogie board everything is white on black…
The only button on the tablet is named “erase” – and that’s what the button does: the whole display flashes two times, one white, and then black and everything is back to where we started. You cannot save. You just press erase and start over. It’s truly a replacement for post-it-notes…
Of course there’s a battery inside, and it’s said to hold for tens of thousands of erases. You cannot change the battery when it’s empty, but on the other hand this gadget is less than 30 Euros and it does look like you can break it up and try your best to exchange the battery yourself. Since the battery isn’t needed to display anything I don’t think I will run out of juice just yet.
Am kommenden Freitag soll das Space Shuttle Endeavour zum letzen Mal und ein Space Shuttle zum vorletzten Mal abheben. Da will man dabei sein 🙂
Ich habe glücklicherweise gerade die Herren (und Damen?) von SpaceLiveCast entdeckt. Offenbar machen die schon eine ganze Weile Livestreams zu den verschiedenen Raumfahrt-Events.
P.S.: Wenn ich einen Wunsch frei hätte, wäre das, dass die Seite einen Video Podcast Feed anbietet….(wird Hilfe benötigt?)
Configuring your favourite Editor on OSX (or Linux, or anywhere else) is important – since nano is my editor of choice I wanted to use it’s syntax highlighting capabilities. Easy as pie as it turned out:
I started with a .nanorc file from this guy and modified it to recognize some of my frequent file-types (like .cs files).
You can download my nanorc.tar – just extract it and put it into your user home directory.
Source 1: http://talk.maemo.org/showthread.php?t=68421
Source 2: http://www.nano-editor.org/dist/v2.2/nano.html#Nanorc-Files
Source 3: nanorc.tar
It’s been some
months years since the once Microsoft Research Project got public and Microsoft started offering it’s great Photosynth service to the public.
I’ve been using the Microsoft panoramic and Photosynth tools for years now and I tend to say that they are the best tools one can get to create fast, easy and high-quality panoramic images.
There is photosynth.net to store all those panoramic pictures like this one from 2008:
The photosynth technology itself contains several other interesting technologies like SeaDragon which allows high quality image zooming on current internet connection speeds.
This awesome technology is as of now available on the iPhone (3GS and upwards) and it’s better than all the other panoramic tools I’ve used on a phone.
Oh boy, it seems that Apple just screwed up big time when it comes to data privacy. Obviously everytime someone attaches an iOS device like the iPhone to a PC or Mac and it does a backup run this backup includes the location data of that iPhone of the last several months. Impressive logging on the one hand and a shame that they did not talk about that in public upfront on the other hand.
There’s a great tool available on GitHub which uses OpenStreetMap to visualize the logged data – it creates a quite impressive graphical representation of where I was the last 6 months…
Irgendwie werden es auch privat immer immer mehr Daten – mit immer zunehmender Geschwindigkeit… Alle paar Jahre tausche ich bei uns im Haushalt die Festplatten/Speicherlösung komplett aus – was zwar immer wieder mal eine Investitions bedeutet, gleichzeitig aber auch dafür sorgt dass Daten nicht irgendwelchen ungünstigen mechanischen, chemischen oder magnetischen Effekten zum Opfer fallen… Ja so etwa alle zwei Jahre wird alles einmal umkopiert… Das dauerte beim letzten Mal zwar gut eine Woche, aber naja so ist das eben…
Aus vielerlei Grund haben wir auch für einen Haushalt recht viel Bedarf an Speicherplatz – teilweise wohl auch weil meine Frau Photographin ist – aber ich als “werf-nix-weg”-Typ werd da auch einen guten Anteil dran haben…
Herr über alle unsere Festplatten (kein Witz, die Rechner bei uns haben ihre Festplatten eigentlich nur um booten zu können) ist seit jeher ein einzelner Rechner welcher ebenso alle paar Jahre komplett ausgetauscht wird. Dieser Rechner verwaltet im Moment zwischen 12-15 Festplatten verschiedener Größe – Hauptarbeit wird zur Zeit durch drei separate (gewachsene) RAID-5 Volumes erledigt…
Nebenbei: Nein ich kann/will da kein RAID-6 fahren ohne entweder Linux zu verwenden (was aus verschiedenen Gründen nicht geht) oder einen Hardware-Controller zu verwenden, was nach einschlägigen Erfahrungen querbeet durch alle möglichen Hardware RAID Controller ausfällt.
Dem ganzen Festplattenstapel liegt dann ein Standard-PC mit Windows Server 2008 zugrunde – zum einen weil ich so eine Lizenz noch herumliegen hatte und zum anderen weil ich in über 10 Jahren File-Server Erfahrungen sammeln noch nie auch nur ein Byte unter Windows verloren habe. Zusätzlich habe ich einen riesigen Haufen Software welche Windows-only ist ud sozusagen ständig laufen muss um Sinn zu machen (Mail-Server Puffer, Newsserver Mirror, Musik und Video Streaming Server, Medienbibliothek, Videorekorder,…
Diese drei großen RAID Volumes schnappt sich dann Truecrypt und ver- und entschlüsselt zuverlässig vor sich hin – im Endeffekt gibt es kein Byte Daten im Haushalt welches nicht verschlüsselt wäre. Gut für uns.
So ein RAID verhindert nun ja aber nicht dass dennoch oben genannte ungünstige Effekte eintreten und man mal eine oder mehrere Defekte zu beklagen hat. Im Normalfall tauscht man die defekte Festplatte, resynct das RAID und alles funktioniert weiter ohne dass man Daten verloren hätte. Allerdings ist das ja kein Backup. Das ist nur eine erste Absicherung gegen mögliche Defekte.
Getreu folgendem kurzen Musikstück:
… ist ein RAID eben kein Backup. Backups erledigt bei mir eine Sammlung von Scripten welche jeweils in festen Abständen Vollbackups und Differenz-Backups erstellt. Da kommt dann ein Haufen 1 Gbyte großer Dateien raus welche dann anschliessend per RSync in mühevoller (und dank funktionierendem QoS unbemerkt) Arbeit außer Haus geschafft werden. Die Komplett-Backups dauern aufgrund der großen Menge einfach ewig lang und lassen sich recht einfach dadurch beschleunigen dass man sozusagen das Backup physisch auf einer externen Festplatte zum Server trägt…die Differenz-Backups sind dann meist immer recht flott durchgelaufen. Speicherplatz im Internet wird ja auch immer billiger und so haben wir auch immer ein gutes Off-Site Backup unserer Daten…
Für Windows gibt es neben den üblichen Cygwin Ports von rsync auch eine gute GUI Version namens DeltaCopy. Das Ding kopiert zuverlässig und auch wenn mal der DSL Router rebootet oder hängt nimmt er selbständig die Kopierarbeit wieder auf sobald Netz wieder verfügbar ist.
Damit DeltaCopy seine Daten irgendwo abladen kann wird auf der Gegenstelle natürlich ein rsync Server vorrausgesetzt. Die Konfiguration eines solchen ist nicht sonderlich kompliziert – im Grunde muss man nur rsync installieren und die rsyncd.conf Datei anpassen. Zusätzlich dazu muss man eine Konfigurationsdatei anlegen in welchem nach dem Schema “Benutzername:Passwort” entsprechend die Nutzeraccounts angegeben werden – das wars eigentlich schon. Rsync ist sehr robust und vor allem auch gut für geringere Bandbreiten geeignet. Wenn sich an einer Datei nur wenige Bytes geändert haben müssen auch nur die geänderten Bytes übertragen werden.
Source 1: http://www.speichergurke.de
Source 2: http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp
Source 3: http://de.wikipedia.org/wiki/Rsync
I am using the Apple AirPlay technology for several years now… after it got implemented into iOS it’s just fantastic to have the option to have whatever sound source I want to playing loud and clear in any room I want to…
Okay it’s not quite as sophisticated as the sonos solution regarding the control of multiple music sources in multiple rooms but it get’s the job done in an apartment.
So back to the topic: Apple integrated the AirPlay technology into their wireless base station “AirPort Express”. Basically AirPlay is a piece of software which receives an encrypted audio stream over the network and outputs the stream to the SPDIF or audio jack.
Back in 2005 there already was an emulator of this protocol called “Fairport” but Apple decided to encrypt the AirPlay traffic. This led to the problem that the encryption key was unkown because it’s baked into the AirPort Express firmware. And this is where the good news start:
“My girlfriend moved house, and her Airport Express no longer made it with her wireless access point. I figured it’d be easy to find an ApEx emulator – there are several open source apps out there to play to them. However, I was disappointed to find that Apple used a public-key crypto scheme, and there’s a private key hiding inside the ApEx. So I took it apart (I still have scars from opening the glued case!), dumped the ROM, and reverse engineered the keys out of it.”
So to keep things short: Someone got an AirPort Express, dumped the firmware, extracted the AirPlay encryption keys and wrote an emulator of the AirPlay protocol which uses the key. Voilá!
ShairPort is available in source code on the site of the guy and obviously it’s unsure if Apple will react by changing the encryption key in the future. But for the time being it works as advertised:
I took one of my computers and followed the instructions to update perl, install Macports and then run ShairPort. So when ShairPort is run it looks not as appealing as expected:
Notably it uses IPv6 to communicate between iTunes and ShairPort… Oh I almost forgot to show how it looks in iTunes:
On another side note: It works on Linux, Windows and Mac OS X 🙂
Using OS X for the daily work is getting easier every day. And most of the time I am doing work using the Terminal.app.
- Edit /etc/bashrc and add some alias and color definitions
- alias ll=”ls -hfG”
- alias la=”ls -ahfG”
- export LSCOLORS=fxfxcxdxbxegedabagacad
- custom color schemes can be defined using the lscolors tool
- install screen (using MacPorts for example) and setup a ~/.screenrc
I’ve worked on my little holiday project for a while now and it’s making great progress. Since logging is working for almost two weeks now I got some data that should be visualized. One main goal of the project is to have a great UI to browse the sensor data.
The logging server now included an internal http server which serves some pages and RESTful services already. One of those services is the sensor data service which can be asked to output JSON formatted sensor data. If you take that data using jQuery and the flot jQuery plugin you’ll get something like that:
This EzControl XS1 device is a complex thing. And currently I am playing with more than 10 sensors and more than 10 actuators. Since poking around with such a device will most certainly lead to a condition where that configuration might get lost (like a power down for more than 30 minutes).
Therefore I was in need of a backup and restore tool. Because there isn’t one I had to write one myself. Here it is:
My tool is available as opensource as part of the h.a.c.s. toolkit here. Enjoy!
Source 1: https://github.com/bietiekay/hacs/wiki/H.a.c.s.-toolkit
Source 2: http://github.com/bietiekay/hacs/
My holiday project is progressing: Today it was hardware delivery day!
So this is the hardware which is ready to be used:
- 1x EzControl XS1 controller
- 2x Temperature and Humidity sensor
- 8x Remote Power Switch
The EzControl XS1 is easy to use as far as I had the time to give it a try. After the network setup the XS1 offers a simple web interface and REST service. Built upon that REST service there is also a configuration application and a visualization application available. Those two applications are apparently built using the GWT framework.
I poked around a bit with the sensor and actor configuration screens and everything just worked. Those applications are great for the easy tasks. And for everything else hacs is what is going to be the tool of choice (to be written).
Source 1: http://www.ezcontrol.de
Source 2: http://github.com/bietiekay/hacs
Source 3: http://code.google.com/intl/de-DE/webtoolkit/overview.html
Hurray! One of those EzControl XS1 plus some sensors and actors is on the way to me. So I can finally start the little holiday project which will be called “HACS” (Home Automation Control Server).
The source code and documentation repository is up on GitHub as of now – you can access it here: https://github.com/bietiekay/hacs
If you are interested in working on that project – drop a comment.
Thank goodness I can uninstall X-Lite! At sones we are using a SIP based telephony solution. And therefore some times a SIP softphone application is needed along with the obligatory hardware SIP telephones. Till today the only half-working software I knew for that task was X-Lite. But a colleague told me today that there is a better software which not even looks better but also works better than X-Lite.
It’s called “Ekiga” and it’s a GTK based open source application which can run on Windows and Linux. It looks clean and therefore nice and works great.
A special tip from me: Abort the Welcome Wizard because the only thing it does is registering you with ekigas’ own services.
Es ist ja nun schonwieder einige Zeit her dass ich etwas über meine CB-Funk Software namens “FFN-Switcher” geschrieben habe. Nun ist es immerhin mal wieder soweit dass ich zeit gefunden habe mich mit einigen Bugfixes zu beschäftigen.
Gleichzeitig habe ich den Sourcecode von meinem privaten Subversion Repository auf den öffentlich zugänglichen GitHub Dienst hochgeladen. Dort kann der Sourcecode und was noch viel wichtiger ist: die Bug- und Wunschliste abgerufen und editiert werden.
Natürlich gab es in der Zwischenzeit auch einige Bugfixes. Sodass mittlerweile Version 111 online steht und über die automatische Updatefunktion abgerufen werden kann.
Hurray! Finally the 2.8 version of Mono – the platform independent open source .NET framework is available as of today. I finally don’t have to recompile the trunk every now and then to get my bits running
The Major Highlights according to the release notes are:
- C# 4.0
- Defaults to the 4.0 profile.
- New Garbage Collection engine
- New Frameworks:
- Parallel Framework
- Threadpool exception behavior has changed to match .NET 2.0
- potentially a breaking change for a lot of Mono-only software
- See information below in the "Runtime" section.
- New Microsoft open sourced frameworks bundled:
- Managed Extensibility Framework
- ASP.NET MVC 2
- System.Data.Services.Client (OData client framework)
- Large performance improvements
- LLVM support has graduated to stable
- Use mono-llvm command to run your server loads with the LLVM backend
- Preview of the Generational Garbage Collector
- Version 2.0 of the embedding API
- WCF Routing
- .NET 4.0’s CodeContracts
- Removed the 1.1 profile and various deprecated libraries.
- OpenBSD support integrated
- ASP.NET 4.0
- Mono no longer depends on GLIB
Oh – they even linked my benchmark article.
In the last 10+ years I was fiddling with different home automation concepts. Mostly without broad use cases because at that time no one seemed to be interested in having sensors and actors like crazy at home. In fact not that many people seem to care these days.
Having more and more hardware and software around us creates the use cases for a broader audience people like me have for 10+ years. Mainstream is a bitch for nerds
That said I found a nice plastic box I want to use in a winter project. This plastic box is called “EzControl XS1”. It comes with several visible and “invisible” interfaces.
The visible and obvious ones are: power, 100 mbit ethernet, sd card slot. So it takes some power and does something on the network. The not so obvious and therefore “invisible” interfaces are the most interesting ones: the EzControl XS1 comes with the ability to send and receive on 433 Mhz and 868 Mhz.
Yes that are the ranges used by switchable and dimable power sockets, temperature sensor and AMR. The EzControl XS1 is not that cheap (coming at 189 Euros for the base version and additional 65 Euros per upgrade option). I do not own one yet so it’s the plan to acquire at least one and start of with dimable power sockets and add more sensors and actors on the way
One great feature of the EzControl XS1 is the embedded WebServer with which the users application (the one I want to write) can interact using a HTTP/JSON Protocol. Oh dear: Sensor data and Actor control using JSON. How great is that!
There is some example code available (even a proprietary iPad/iPhone client) but since I want to have some custom features I do not currently see to be available in software I am going to write a set of tools which will get and protocol sensor data and run scripts to controls actors. Oh it’ll be all available as open source (license not yet chosen).
P.S.: If some one from Rose+Herleth is reading this and wants to help – send me a test unit
Source 1: http://www.ezcontrol.de (in german though)
Source 2: http://en.wikipedia.org/wiki/Automatic_meter_reading
Source 3: http://www.ezcontrol.de/content/view/12/31/
I am a huge fan of the Windows Live Writer. It’s been some years now since Microsoft made this free tool available to bloggers who want to blog on Windows. And in a bold move Microsoft announced the other week that they will be moving all Windows Live Spaces weblogs (a free weblog hosting service) to WordPress.
In an accompanying step they just released the 2011 version of the Windows Live Writer. Actually I think it’s a shame that there is no comparable tool on Mac OS X … which is quite unusual since those types of tools in that quality are more common on the apple platform.
The new Window Live Writer 2011 comes with the Ribbon UI already known from Office 2007 and 2010 (and 2011 now).
There’s a great tool available to create impressive visualizations of source code repositories:
“Software projects are displayed by Gource as an animated tree with the root directory of the project at its centre. Directories appear as branches with files as leaves. Developers can be seen working on the tree at the times they contributed to the project.
I often have to draw concept diagrams and until now I had to use MindMap tools and tools like Visio. And up until now it wasn’t that much fun… but first things first, what’s a Concept Map?
“A concept map is a diagram showing the relationships among concepts. They are graphical tools for organizing and representing knowledge.
Concepts, usually represented as boxes or circles, are connected with labeled arrows in a downward-branching hierarchical structure. The relationship between concepts can be articulated in linking phrases such as "gives rise to", "results in", "is required by," or "contributes to".”
For example a concept map might looks like this:
So I found a tool called “IHMC CmapTools” – a great package of software available for Windows, Mac OS X and Linux. And this tool makes it so much easier to create impressive and expressive concept maps. It’s freeware and can be used even for commercial purposes.
It’s been some time since I’ve written about a Visual Studio Color Theme Generator. And obviously since then a lot happened in the world of customization tools.
The website studiostyles.info is there to help the day with a lot of previewable Visual Studio styles. Even better: all styles can be exported for Visual Studio 2005, 2008 and 2010.
For Visual Studio 2010 you get a .vssettings file which can be imported into Visual Studio using the Tools->Import and Export Settings… menu item.
For Visual Studio 2010 there are additional color styling options available. Microsoft offers a plugin for Visual Studio 2010 called Visual Studio Color Theme Editor. Using this tool everything else can be color customized. So you can have something like that:
If you want to create a (mountable, bootable) image of your local hard disk just use that small and cool tool Disk4vhd
Every once in a while you download some code and fire up your Visual Studio and find out that this particular solution was once associated to a team foundation server you don’t know or have a login to. Like when you download source code from CodePlex and you get this “Please type in your username+password for this CodePlex Team Foundation Server”.
Or maybe you’re working on your companies team foundation server and you want to put some code out in the public. You surely want to get rid of these Team Foundation Server bindings.
There’s a fairly complicated way in Visual Studio to do this but since I was able to produce unforseen side effects I do not recommend it.
So what I did was looking into those files a Visual Studio Solution and Project consists of. And I found that there are really just a few files that hold those association information. As you can see in the picture below there are several files side by side to the .sln and .csproj files – like that .vssscc and .vspscc file. Even inside the .csproj and .sln file there are hints that lead to the team foundation server – so obviously besides removing some files a tool would have to edit some files to remove the tfs association.
So I wrote such a tool and I am going release it’s source code just beneath this article. Have fun with it. It compiles with Visual Studio and even Mono Xbuild – actually I wrote it with Monodevelop on Linux 😉 Multi-platform galore! Who would have thought of that in the founding days of the .NET platform?
So this is easy – this small tool runs on command line and takes one parameter. This parameter is the path to a folder you want to traverse and remove all team foundation server associations in. So normally I take a check-out folder and run the tool on that folder and all its subfolders to remove all associations.
So if you want to have this cool tool you just have to click here: Sourcecode Download
So finally after years and years of hope and nerdy ideas I am able to hold a tablet device in my own hands and it’s not only as good as Picards tablet was back in that great “Star Trek: Next Generation” series, it’s better.
Of course I had to import that particular iPad from the U.S. (thanks Alex!) – actually it was the first time I imported something that expensive. Beside some fun with the shipping company everything went fine. Since Apple just announced to delay the launch of the iPad in Europe for a month it’s nice to have a gadget just a few weeks after it was available in the U.S.
Developing software is hard work – especially when you target several operating systems. One task that you have to perform quite often would be to deploy a new installation of an operating system as fast as possible on a test machine.
Doing this with Windows is easy – you can use the Windows Deployment Services to bootstrap Windows onto almost every machine which can boot over ethernet using PXE. Everything needed to make WDS work on a Windows Boot-Image is located on that image. Since it’s that easy I won’t dive into more detail here.
What I want to show in greater detail is how you can use WDS to deploy even Linux over your network.
Step 1: Get PXELINUX
What’s needed to boot Linux over a network is a dedicated PXE Boot Loader. This one is called PXELINUX and can be downloaded here.
“PXELINUX is a SYSLINUX derivative, for booting Linux off a network server, using a network ROM conforming to the Intel PXE (Pre-Execution Environment) specification.”
On the homepage of PXELINUX is also a short tutorial which files you need and where to copy them.
Step 2: Setup WDS with PXELINUX
I suppose you got your WDS Installation up and running and you are able to deploy Windows. If that’s the case you can go to your WDS Server Management Tool and right-click on the server name – in my case “fileserver.sones”. If you select “Properties” in the context menu you would see the properties windows like in the screenshot below:
You have to change the Boot-Loader from the standard Windows BootMgr to the newly downloaded PXELINUX bootloader. Since this bootloader comes with it’s own set of config files you can edit this config file to allow booting into Windows.
The first entry I made into the boot menu of the PXELINUX boot loader is the “Install Windows…” entry. Since the first thing the users will see after booting is the PXELINUX loader menu they need to be able to continue to their Windows Installation. Since this Windows Installation cannot be handled by the PXELINUX loader you have to define a boot menu entry which looks a lot like this:
MENU LABEL Install Windows…
To add OpenSuSE to the menu you would add an entry looking like this:
MENU LABEL Install OpenSuSE 11.x
append initrd=/Linux/opensuse/initrd splash=silent showopts
The paths given in the above entry should be altered according to the paths you’re using in your installation. I took the /Linux/opensuse/ files from the network install dvd images of OpenSuSE.
That’s basically everything there is about the installation of Linux (Debian works accordingly) over PXE and WDS.
And finally this is what it should look like if everything worked great:
Oh what a nice n3rd toy this would be. Rumors say it will be available soon for under $30. And for those who right now think: “What the hell is this?” – This is a coffee mug in the shape of a quite expensive canon lens. In fact I already heard of that idea more than a year ago and wrote about it here. At this time there were only hopes that it would be produced.
That’s great news for everyone interested in science and history. As it turns out Google and PopSci just made their entire 137-year archive available online… good times!
“We’ve partnered with Google to offer our entire 137-year archive for free browsing. Each issue appears just as it did at its original time of publication, complete with period advertisements. It’s an amazing resource that beautifully encapsulates our ongoing fascination with the future, and science and technology’s incredible potential to improve our lives. We hope you enjoy it as much as we do.”
At sones I am involved in a project that works with a piece of hardware I wanted to work with for about 3 years now: the Microsoft Surface Table.
I was able to play with some tables every now and then but I never had a “business case” which contained a Surface. Now that case just came to us: sones is at the CeBIT fair this year – we were invited by Microsoft Germany to join them and present our cool technology along with theirs.
Since we already had a graph visualisation tool the idea was to bring that tool to Surface and use the platform specific touch controls and gestures.
The good news was that it’s easier than thought to develop an application for Surface and all parties are highly committed to the project. The bad news is that we were short on time right from the start: less than 10 days from concept to live presentation isn’t the definition of “comfortable time schedule”. And since we’re currently in the process of development it’s a continueing race.
Thankfully Microsoft is committed to a degree they even made it possible to have two great Surface and WPF ninjas who enable is to get up to speed with the project (thanks to Frank Fischer, Andrea Kohlbauer-Hug, Rainer Nasch and Denis Bauer, you guys rock!).
I was able to convice UID to jump in and contribute their designing and user interface knowledge to our little project (thanks to Franz Koller and Cristian Acevedo).
During the process of development I made some pictures which will be used here and there promoting the demonstration. To give you an idea of the progress we made here’s a before and after picture:
I think everyone did a great job so far and will continue to do so – a lot work to be done till CeBIT! 🙂
Since sones will be at some community events, conferences and trade shows this year we thought it might be a good idea to have some hardware to document these events.
Since we wanted to have video and we did not want to cope with the rather complex subject of DSLRs we bought a full-hd-camcorder.
As you may know, my team and I are developing a graph database. A graph database is a database which is able to handle such things as the following:
So instead of tables with rows and columns, a graph database concentrates on objects and the connections between them and is therefore forming a graph which can be queried, traversed, whatever-you-might-want-to-do.
Lately more and more companies start realizing that their demand for storing unstructured data is growing. Reflecting on unstructured data, I always think of data which cannot single-handedly be mapped in columns and rows (e.g. tables). Normally complex relations between data are represented in relation-tables only containing this relational information. The complexity to query these data structures is humongous as the table based database needs to ‘calculate’ (JOINs, …) the relations every time they are queried. Even though modern databases cache these calculations the costs in terms of memory and cpu time are huge.
Graph databases more or less try to represent this graph of objects and edges (as the relations are called there) as native as possible. The sones GraphDB we have been working on for the last 5 years does exactly that: It stores and queries a data structure which represents a graph of objects. Our approach is to give the user a simple and easy to learn query language and handle all the object storage and object management tasks in a fully blown object oriented graph database developed from the scratch.
Of course the user can choose between different ways to access the database test instance (like SOAP and REST) but the one we just released only needs a browser.
The sones GraphDB WebShell – as we call it – resembles a command line interface. The user can type a query and it is instantly executed on the database server and the results are presented in either a xml, json or text format.
Granted – the interested user needs to know about the query language and the possible usage scenarios. Everyone can access a long and a short documentation here.
Today was Linux-Distribution-ISO-Install-Day. And it turned out that the only existing external DVD drive was fubar.
So what to do? We had a spare USB stick and it turns out that you can quite easily convert that USB stick into a bootable Linux-Distribution-Install-USB-Stick. Awesome!
Just download the tool called “UNetbootin”, start it and you can turn virtually any ISO Distribution Image into an USB Stick that boots and installs that ISO:
After hitting <return> the folder will be a shortcut to the Windows 7 Administration GOD Mode. Enjoy. (Thanks Damir)
I had the task to make my Outlook Task List appear on my iPhone. As everyone knows Apple did not do anything about todo lists or tasks on their phone… well there’s an app for that: Most of the task applications on the iPhone use Toodledos services to sync task lists with the desktop.
To sync the Toodledo service with the desktop you need another tool. This tool uses your Toodledo account and your locally running Outlook to sync between both. So this little desktop sync tool needs access to the Outlook data: This means you will maybe be bugged by Outlook that some program wants to have access to the data. You can allow it for a number of minutes but not forever.
Okay one solution would be to install appropriate antivirus tools to suit the operating systems security needs. Because this wasn’t a solution in my case I needed something more sophisticated to solve the problem.
Now that’s the point where “Advanced Security for Outlook” from MapiLab comes into play. This Outlook Plugin extends Outlooks Security Dialog and adds things like “always allow”:
Thanks to a podcast I found a great software for my iPhone and iPod touch. It’s a small tool which does cost less than 3 Euro and it’s served by a server tool which runs on Windows and Mac OS X.
It’s called Air Video and it’s frikin’ awesome! ™
What you do is you install the server software and point it to all your directories / drives that might contain video material. You then take your iPhone and install the client app. If you configured the server to be available over the internet you can now connect from anywhere you want using a pass-pin (which is generated) and a password (which is set by you). And by “from anywhere” they mean “anywhere”. WLAN or 3g didn’t make any difference in my test. You start the client, point to a video file and most of the time you are asked if you a) want to directly play is (if the file is ipod-compatible) or b) if you want to live-convert it and play it (when the file isn’t compatible and needs to be re-encoded live for you) or c) if you want to add the file to a conversion queue which will off-line convert the video for you.
In terms of “finding your video” it does look like this:
Simple, eh? Taping a video will bring up this screen:
As I said – Play directoy, Play with Live Conversion and Offline-Conversion-Queue…
It did work with EVERY Video I tried. When I tried Full-HD Movies my serving PC wasn’t able to handle the load but everyhing in SD worked great which is perfect for me.
Therefore I can highly recommend this tool – it really does work better than anything I’ve seen before.
I am proud to anounce that there’s a video publicly available which shows parts and projects Microsoft Research is working on currently. It’s great to see theses projects, concepts and ideas become publicly available one by one:
“Craig Mundie, chief research and strategy officer of Microsoft, presents “Rethinking Computing,” a look a how software and information technology can help solve the most pressing global challenges we face today. Part of UW’s Computer Science and Engineering’s Distinguished Lecture Series, Mundie demonstrates a number of current and future-looking technologies that show how computer science is changing scientific exploration and discovery in exciting ways. He discusses the role of new science in solving the global energy crisis, and answer questions from the audience.”