Scaling Linux: Perfomance Tools and Measurements

 

If you ever experienced a missmatch between the performance you expected from a server or application running on Linux you probably started to debug your way into it why the applications performance is not on the expected levels.

With Linux being very mature you get an enormous amounts of helpers and interfaces to debug the performance aspects of the operating system and the applications.

Want to see proof? Here – a map of almost all the thingies and interfaces you got:linuxperftools

Thankfully Brendan Gregg put together a page with videos and further links to drill into those interfaces and methods above.

Source: http://www.brendangregg.com/linuxperf.html

document your REST interfaces with style: Swagger

Swagger is a specification and complete framework implementation for describing, producing, consuming, and visualizing RESTful web services. The overarching goal of Swagger is to enable client and documentation systems to update at the same pace as the server. The documentation of methods, parameters, and models are tightly integrated into the server code, allowing APIs to always stay in sync. With Swagger, deploying managing, and using powerful APIs has never been easier.”

Bildschirmfoto 2014-03-15 um 22.35.09

Source 1: https://helloreverb.com/developers/swagger
Source 2: https://github.com/wordnik/swagger-core
Source 3: http://petstore.swagger.wordnik.com/#!/pet

“Compressing” JSON to JSON

JSON_logo
JSON Logo

The internet and all those browsers and javascript applications brought data structures that are pretty straight-forward. One of them is JSON.

The wikipedia tells about JSON:

“JSON (/ˈdʒeɪsɒn/ JAY-soun, /ˈdʒeɪsən/ JAY-son), or JavaScript Object Notation, is an open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs. It is used primarily to transmit data between a server and web application, as an alternative to XML.”

Unfortunately complex JSON can get a bit heavy on the structure itself with over and over repetitions of data-schemes and ids.

There’s RJSON to the rescue on this. It’s backwards compatible and makes your JSON more compressible:

“RJSON converts any JSON data collection into more compact recursive form. Compressed data is still JSON and can be parsed with JSON.parse. RJSON can compress not only homogeneous collections, but also any data sets with free structure.

RJSON is single-pass stream compressor, it extracts data schemes from document, assign each schema unique number and use this number instead of repeating same property names again and again.”

Of course this is all open-source and you can get your hands dirty here.

Source 1: http://en.wikipedia.org/wiki/JSON
Source 2: http://www.cliws.com/e/06pogA9VwXylo_GknPEeFA/
Source 3: https://github.com/dogada/RJSON

full text transcripts of the Apple World Wide Developer Conference (wwdc)

Since I’ve become sort of an iOS developer lately I had my fair share of WWDC recordings to get started with this whole CocoaTouch and Objective-C development stuff.

Now a tool that is pretty handy is a this website that offers a full-text transcript search of all WWDC recordings. Awesome!

Bildschirmfoto 2013-11-02 um 00.54.36

Source 1: http://asciiwwdc.com/
Source 2: http://miataru.com/ios

a kilobyte of javascript – js1k

What do you think can you do with 1 kilobyte of javascript? Not a lot you might think. In fact it’s quite a lot!

js1k

Similar to the 4k and 64k demo awards now there is a 1k javascript competition:

“This is a competition about JavaScript scripts no larger than 1k. Starting out as a joke, the first version ended with a serious amount of submissions, prizes and quality.”

So what can you do with 1k of javascript? A lot! Click your way through the demos on the js1k.com site and find a lot like this:

Bildschirmfoto 2013-05-04 um 20.50.54

Source 1: http://js1k.com/
Source 2: http://scene.org/
Source 3: http://awards.scene.org/nominees.php?cat=9

Adobe Photoshop version 1 source code

It’s becoming a fashion lately to release the source code of older but legendary commercial products to the public. Now Adobe decided to gift the source code of their flagship product Photoshop in it’s first version from 1990 to the Computer History Museum.

splashscreen

“That first version of Photoshop was written primarily in Pascal for the Apple Macintosh, with some machine language for the underlying Motorola 68000 microprocessor where execution efficiency was important. It wasn’t the effort of a huge team. Thomas said, “For version 1, I was the only engineer, and for version 2, we had two engineers.” While Thomas worked on the base application program, John wrote many of the image-processing plug-ins.”

Source: http://www.computerhistory.org/atchm/adobe-photoshop-source-code/

Automated Picture Tank and Gallery for a photographer

Since my wife started working as a photographer on a daily basis the daily routine of getting all the pictures off the camera after a long day filled with photo shootings got her bored quickly.

Since we got some RaspberryPis to spare I gave it a try and created a small script which when the Pi gets powered on automatically copies all contents of the attached SD card to the houses storage server. Easy as Pi(e) – so to speak.

IMG_2322

So this is now an automated process for a couple of weeks – she comes home, get’s all batteries to their chargers, drops the sd cards into the reader and poweres on the Pi. After it copied everything successfully the Pi sends an eMail with a summary report of what has been done. So far so good – everything is on our backuped storage server then.

Now the problem was that she often does not immediately starts working on the pictures. But she wants to take a closer look without the need to sit in front of a big monitor – like taking a look at her iPad in the kitchen while drinking coffee.

So what we need was a tool that does this:

  • take a folder (the automated import folder) and get all images in there, order them by day
  • display an overview per day of all pictures taken
  • allow to see the fullsized picture if necessary
  • work on any mobile or stationary device in the household – preferably html5 responsive design gallery
  • it should be fast because commonly over 200 pictures are done per day
  • it should be opensource because – well opensource is great – and probably we would need to tweak things a bit

Since I did not find anything near what we had in mind I sat down this afternoon and wrote a tool myself. It’s opensourced and available for you to play with it. Here’s a short description what it does:

It’s called GalleryServer and basically is an embedded http server which takes all .jpg files from a folder (configurable) and offers you some handy tool urls which respons with JSON data for you to work with. I’ve written a very small html user interface with a bit of javascript (using the great html5 kickstart) that allows you to see all available days and get a nice thumbnail overview of each day – when you click on it it opens the full-size image in a new window.

It’s pretty fast because it’s not actively resizing the images – instead it’s taking the thumbnail picture from the original jpg file which the camera placed there during storing the picture. It’s got some caching and can be run on any operating system where mono / .net is available – which is probably anything – even the RaspberryPi.

Source 1: my wifes page
Source 2: 99lime html5 kickstart boilerplate
Source 3: https://github.com/bietiekay/GalleryServer

Security Engineering — The Book

The second edition of the book “Security Engineering” by Ross Anderson is available as a full download. It’s quite a reference and a must-read for anybody with an interest in security (which for example all developers should have).

“When I wrote the first edition, we put the chapters online free after four years and found that this boosted sales of the paper edition. People would find a useful chapter online and then buy the book to have it as a reference. Wiley and I agreed to do the same with the second edition, and now, four years after publication, I am putting all the chapters online for free. Enjoy them – and I hope you’ll buy the paper version to have as a conveient shelf reference.”
book2coversmall

Source 1: http://www.cl.cam.ac.uk/~rja14/book.html