“are you still watching?” – tailfix

drop in replacement for tail -F that asks you if you are still watching

Just like Laura I am also was having a moment when I stumbled across tailflix.

For those not understanding the reference: At the end of an episode you’ve watched on Netflix you will be shown another one, and so on, and so on. Until if you have not touched the remote at all for several episodes Netflix will ask you “Are you still watching?”.

serve live interactive Jupyter notebooks

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.

voilá

Voila serves live Jupyter notebook including Jupyter interactive widgets.

Unlike the usual HTML-converted notebooks, each user connecting to the Voila tornado application gets a dedicated Jupyter kernel which can execute the callbacks to changes in Jupyter interactive widgets.

https://github.com/QuantStack/voila

Look me in the eye

As tools get better artists really polish out what technology enables them to do with computers nowadays.

This eye above made me awww. It’s from a user “ChrisJones” in the Blender Forum who posted his progress on modelling a human head in upmost detail.

Go there and be awwd as well. And while you’re there grab a copy of Blender and give it a go.

clone files from and to the cloud

Rclone is a command line program to sync files and directories to and from:

Alibaba Cloud (Aliyun) Object Storage System (OSS), Amazon Drive, Amazon S3, Backblaze B2, Box, Ceph, DigitalOcean Spaces, Dreamhost, Dropbox, FTP, Google Cloud Storage, Google Drive, HTTP, Hubic, Jottacloud, IBM COS S3, Koofr, Memset Memstore, Mega, Microsoft Azure Blob Storage, Microsoft OneDrive, Minio, Nextcloud, OVH, OpenDrive, Openstack Swift, Oracle Cloud Storage, ownCloud, pCloud, put.io, QingStor, Rackspace Cloud Files, Scaleway, SFTP, Wasabi, WebDAV, Yandex Disk, The local filesystem

rclone.org

Whenever you would need to move large and/or many files between cloud storages you should give rClone a try.

technical visualization tools

There’s so much happening in this field as visualizations become more powerful and easier to create.

WaveDrom

WaveDrom draws your Timing Diagram or Waveform from simple textual description. It comes with description language, rendering engine and the editor.

WaveDrom editor works in the browser or can be installed on your system. Rendering engine can be embeded into any webpage.

https://wavedrom.com/

MermaidJS

Generation of diagrams and flowcharts from text in a similar manner as markdown.

Ever wanted to simplify documentation and avoid heavy tools like Visio when explaining your code?

This is why mermaid was born, a simple markdown-like script language for generating charts from text via javascript. Try it using our editor.

the future of (speech) podcasts

The Podlove project once again leads the way to improve the experience and the way we interact with knowledge and thoughts. With the most current announcement and introduction of transcription-support for podcasts. Click on it right now and try it yourself.

Fulltext-search. Listening to podcasts by reading them. This-is-amazing!

Transcripts are coming

Transcripts are an incredibly desirable thing to have for podcasts: they allow searching for specific parts, increase searchability by search sites when presented properly and they increase the accessibility of audio content significantly too.

However, transcripts have been considerably difficult to be created and used. Manually created transcripts are costly in terms of time and money and even if you spend the money there has been a lack of technical standards for storing and integrating transcripts into websites in a defined way.

This is now slowly changing: more and more automated speech-to-text systems are becoming available at reasonable costs and they are creating ever better transcripts with more and more languages being supported.
Still, automatic transcripts trail manually created transcripts in terms of accuracy, punctuation and so on but they are increasingly useful when they are primarily used for improving search results or helping you with your internal research when trying to find content in your older episodes.

New services are also coming up to deal with these problems by allowing users to quickly build on automatic transcripts and improve them manually in an assisted fashion. We will soon see a landscape of tools and services that will make creating transcripts easy and cheap enough for more and more podcasters so it’s time to come up with a good integration.

Last but not least, the WebVTT file format has become a de-facto common denominator for passing transcripts along, supporting time codes, speaker identification and a rudimentary set of meta data. While not perfect it’s enough to get a transcript infrastructure up and running and Podlove is leading the way.

from the Podlove Publisher 2.8 announcement

make linux fast (unsecure) again

The CPU/hardware related bugs surfacing the last couple of years have mostly been fixed by adjusting the software that is run. Sometimes only by disabling certain features of a CPU or patching the microcode in the CPU itself.

The issue with this is that by fixing these issues features got disabled and workarounds had been introduced that lowered performance. Dramatically so for some use-cases.

By how much? Well it really depends on your CPU and use-cases. But maybe you want to try yourself. If you want to know the most current parameters to pass to your kernel on boot-up to disable all the performance impacting fixes, go here:

It is not recommended to have this in productive use – as you can imagine. Those bugs where fixed for a reason.

look at the earth

Watch sunlight and weather patterns move across Earth throughout the day, and bask in the glory of our blue marble in real time.

Every 20 minutes (or every hour, you pick), Downlink updates your desktop background with the newest images of Earth.

Choose from 8 different views of Earth, including stunning full disk images from 3 different geostationary satellites.

Downlink is a macOS application which downloads the most current image taken by earth orbitting satellites. So what you see is as accurately depicting reality as technically possible right now.

Interestingly all this data comes from public domain sources. And the makers of the app have documented their sources:

Thanks to NASA, NOAA, JMA, Lockheed Martin, Harris Corporation, ULA, MHI, and everyone else who designed, built, launched, and operate GOES-16, GOES-17, and Himawari-8.
If you’d like to use the sources Downlink is built on top of, here is the file pointing to those resources. Use it, and tell me what you’ve built!

I am using macOS as well as Linux as my main desktop operating systems. My macs are set with Downlink. But what about my Linux machines?

Easy! The file referring to the sources looks like this:

With some simple steps on Linux we can grab the URL we want, download the image and update the desktop background.

Step 1: Getting the URL from this JSON.

Install jq, curl and xargs and in your shell run this:

curl -s http://downlinkapp.com/sources.json | jq -r '.sources[2].url.full' 

This will give you the URL “full” of the 2nd source in the list (Hiwamari-8).

Step 2: Download the image.

curl -s http://downlinkapp.com/sources.json | jq -r '.sources[2].url.full' | xargs -I{} curl -s -o background.jpg {}

By simply adding the xargs and curl call it will take the URL output from the command in front as input to the second curl call and download it. It also will store the file as “background.jpg”.

Step 3: Setting as background image.

Setting the background image of your desktop depends on what desktop software you are using on Linux. Depending on that you will need to look up in the manual how to set the background image from the command line.

But for most use cases there’s a tool that helps. It’s called Variety.

Variety is an open-source wallpaper changer for Linux
Variety is packed with great features, yet slim and easy to use. It can use local images or automatically download wallpapers from Unsplash and other online sources, allows you to rotate them on a regular interval, and provides easy ways to separate the great images from the junk. Variety can also display wise and funny quotations or a nice digital clock on the desktop.

bumps ahead

This website is delivered to you by a single dedicated server in a datacenter in Germany. This server is old.

11:13:58 up 1320 days, 25 min, 2 users, load average: 1.87, 1.43, 1.25

uptime

And I am replacing it. While doing so I am going to take some shortcuts to lower the effort I have to put in for the move.

It will save me 2 days of work. It will mean for you: there might be some interruptions of the services provided by this website (there are more than this page…).

legit programming language

Programs written in legit are defined entirely by the graph of commits in a Git repository. The content of the repository is ignored.
legit is designed so that all relevant information is visible when running git log --graph --oneline.

For example, here is “hello world”:

You can find implementations of legit, as well as some example programs, on GitHub: https://github.com/blinry/legit. The entry in the Esolang wiki is at https://esolangs.org/wiki/Legit.

80s code editor theme

The beginning of the decade saw the continuation of the clothing styles of the late 1970s and evolved into heavy metal fashion by the end. However, it had a lot of changes considering that, this fashion became more and more extravagant during the 80s.

The 80s included things like teased hair, ripped jeans, neon clothing and lots of colours and different designs which at first weren’t accepted for a lot of people.

Popular Culture in the 80s

Do you remember that endless summer back in ’84? Cruising down the ocean-highway with the top down, the wind in our hair and heads buzzing with neon dreams?
No, I don’t remember it either, but with this experimental theme we can go there.

Synthwave’84 theme

transparently migrate data from local file storage to key-value-stores like RIAK KV

About 2 years ago I sat down and wrote a filesystem. Well not from scratch but using the great FUSE (Filesystem in Userspace) framework. I’ve released it as open-source later on Github:

This script acts as glue between a local file storage mount point and RIAK. It is targeted at specific use cases when local mount-points need to be migrated to RIAK without changing the applications accessing those mount point. Think of it as a transparent RIAK filesystem layer with multiple options to control it’s behavior regarding local files.

riakfuse

I had a very specific purpose in mind when I wrote this: There was a local filesystem that got filled up and because of technical restrictions we were not able to resize it or even completely copy it without interrupting the service using the data stored there.

Since we were already using the RIAK Key-Value Database for certain binary loads the idea came up to also utilize this key-value concept for a filesystem.

The idea now is: You have a local filesystem that holds a lot of folders and files already and you want to gradually want to move it to new grounds.

This migration needed to happen with minimal service interruptions assuming that there is constant reading and writing happening.

In this riak-fuse project I’ve written an overlay filesystem that steps between the application and the underlying “old”-filesystem. It looks and behaves identical to the application reading and writing.

But, depending on the mode you have chosen while mounting, every file read will at first be read from the “old”-filesystem and after successfull read stored into the key-value store.

On the next read it will be read from the key-value store directly.

The same applies for writing. Riak-fuse will write either to both, local storage and key-value store or just to the key-value store.

So in a nutshell: Data is slowly but surely on each access transferred over to the key-value store and load + storage space is slowly moving over from the local storage to the key-value store.

To facilitate this there are a lot of options for this script:

This all comes with ready-to-use docker and docker-compose files for you to try out.

Also it might interest you as an extremely simplified example of how to write an actual file system module for FUSE in Python.

Disclaimer: This effectively is my first python script as well as fuse module. Don’t be too hard on your judgement.

Time estimation in software development

I’ve found myself in these spots several times in my life. Either I had to deliver on an estimate or I had to acknowledge an estimation and deal with the outcomes.

If you are involved in anything digital / software this is a recommended piece to read:

Anyone who built software for a while knows that estimating how long something is going to take is hard. It’s hard to come up with an unbiased estimate of how long something will take, when fundamentally the work in itself is about solving something. One pet theory I’ve had for a really long time, is that some of this is really just a statistical artifact.

Why software projects take longer than you think

convert Markdown scribbles to vector drawings

Every task you take
Every meeting you make
Every keyboard you break
Every note you take
I’ll be storing it for you

my text editor

Well that was fun! And indeed: a big portion of my professional daily business is taking place in a text editor taking notes and scribbling ideas and thoughts.

I’ve tried many things but the only way that resonated with me was taking notes in Markdown in a text editor that supports markdown. Currently that editor is Atom.io. Mainly because it is not in the way and quite portable. Runs on Linux, Windows, MacOS.

extra cheesy 80s neon theme

This way – I just took a count – I noted down 364.416 words in the last 1.5 years on my current job (equals to about 46 hours of average speed reading…).

Along side those simple text notes and bullet lists I am doing very simple tables as well as ASCII scribbles in Markdown as well. With the right tools it’s extremely easy and much faster than booting up the Powerpoint or worse.

When you have all in Markdown you then can freely stylesheet away and convert to handy PDF files as well. All even with embedded images if you so desire.

But even if you sit on that treasure trove of Markdown there comes the time when you wish you could convert your scribbles to graphics. Even if it is for the sole reason to not have to draw it again for that fancy Powerpoint slide deck.

You’ve got multiple options to accomplish this:

svgbob is at first a command line tool that got a recent level-up with a proper web-frontend:

When given Markdown it creates graphics. In the picture above the input is on the left and the svgbob output on the right (as SVG).

Markdeep is the alternative. Which of both work for your case depends on that specific case. Knowing and using both properly is the best way.

from now on console emulation will look much better

A modder going by the handle DerKoun has released an “HD Mode 7” patch for the accuracy-focused SNES emulator bsnes. In their own words, the patch “performs Mode 7 transformations… at up to 4 times the horizontal and vertical resolution” of the original hardware.

The results, as you can see in the above gallery and the below YouTube video, are practically miraculous. Pieces of Mode 7 maps that used to be boxy smears of color far in the distance are now sharp, straight lines with distinct borders and distinguishable features. It’s like looking at a brand-new game.

ArsTechnica

Michelin Guide Restaurants in Tokyo

You are likely aware of the existence of the “Michelin Guide“.

Michelin Guides are a series of guide books published by the French tyre company Michelin for more than a century. The term normally refers to the annually published Michelin Red Guide, the oldest European hotel and restaurant reference guide, which awards up to three Michelin stars for excellence to a select few establishments.

Wikipedia

You might also be aware that Tokyo is the city with the highest density of Michelin star rated restaurants. Nice, eh?

A purchase of this guide is recommended in any case but these days people also need something they can intuitively use and which integrates into already existing workflows.

These people, like myself, need a map and maybe more details in a machine readable, filterable spreadsheet.

And as time goes on it might be quite useful to have all the sources that lead to these great tables and maps. Sources that allow you to crawl and grab these information.

A script that crawls Tokyo-based michelin guide establishments and saves it into a JSON file. I personally did this project so I can plan my tokyo trip based on the cheapest and most-renowned restaurants,

Michelin Guide Crawler on GitHub

Swappiness is a thing, as is cache pressure

We know that using swap space instead of RAM (memory) can severely slow down the performance of Linux. So then, one might ask, since I have more than enough memory available, wouldn’t it better to remove swap space completely? The short answer is, No. There are performance benefits when swap is enabled, even when you have more than enough ram.

Why you should almost always add swap space

vfs_cache_pressure – Controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects. (default = 100, recommend value 50 to 200)

swappiness – This control is used to define how aggressive the kernel will swap memory pages. Higher values will increase aggressiveness, lower values decrease the amount of swap. (default = 60, recommended values between 1 and 60) Remove your swap for 0 value, but usually not recommended in most cases.

https://access.redhat.com/solutions/103833

As I’ve now brought up the topic, go ahead and read the complete story at the source.

procedural generated traditional Chinese landscape scrolls

{Shan, Shui}* is inspired by traditional Chinese landscape scrolls (such as this and this) and uses noises and mathematical functions to model the mountains and trees from scratch. It is written entirely in javascript and outputs Scalable Vector Graphics (SVG) format.

https://github.com/LingDong-/shan-shui-inf

This is quite impressive and I am thinking about pushing that into the header of this blog :-) It’s just too nice looking to pass on.

RSS is here. Use it!

RSS aka RDF Site Summary aka Rich Site Summary aka Really Simple Syndication is a standardized web format that works for you.

At least it would work for you if you would use a a tool which would allow you to “subscribe” to RSS feeds from all sorts of websites. These tools are called feed-reader.

The website you are reading this on offers such a link. By subscribing to its feed you will be able to see all content but without having to actually go to each of your subscriptions one by one. That is done by the feed reader. This process of aggregation is it why feed readers are also called aggregator.

Invented exactly 20 years ago this month on the back-end of a feverish dot-com boom, RSS (Real Simple Syndication) has persisted as a technology despite Google’s infamous abandonment with the death of Google Reader and Silicon Valley social media companies trying and succeeding to supplant it. In the six years since Google shut down Reader, there have been a million words written about the technology’s rise and apparent fall.
Here’s what’s important: RSS is very much still here. Better yet, RSS can be a healthy alternative…

RSS is Better Than Twitter

I am using Liferea as my feed reader on desktop and Reeder on all that is iOS/macOS.

I’ve found that by using RSS feeds and not following a pre-filtered timeline I would not “follow” 1000 sources of information but choose more carefully whom to follow.

Some do not offer any feeds – so my decision in these cases is wether or not I would invest the time to create a custom parser for their content to pull in.

After RSS being just another XML format you quickly realize that HTML is just another XML format as well. There are simple ways to convert between both on the fly. Like fetchrss.com or your command-line.

Of course RSS is not the only feed format: ATOM would be another one worth mentioning.

a terminal for Windows

As Windows lately tends to make an effort to stay out of the way as an operating system and user-experience it seems that it regains more attention by developers.

For me this all is quite strange as I’ve personally would prefer switching from macOS to Linux rather than Windows.

But for those occasions you need to go with Windows. There’s a Terminal application now that gives you, well, a good terminal. Try FluentTerminal.

archive your slice of the web

Many use and love archive.org. A service that roams the public internet and archives whatever it finds. It even creates timelines of websites so you can dive right into history.

Have a piece of history right here:

You can have something similar hosted in your own environment. There are numerous open source projects dedicated to this archival purpose. One of them is ArchiveBox.

ArchiveBox takes a list of website URLs you want to archive, and creates a local, static, browsable HTML clone of the content from those websites (it saves HTML, JS, media files, PDFs, images and more).

I’ve done my set-up of ArchiveBox with the provided Dockerfile. Every once in a while it will start the docker container and check my Pocket feed for any new bookmarks. If found it will then archive those bookmarks.

As the HTML as well as PDF and Screenshot is saved this is extremely useful for later look-ups and even full-text search indexing.

if you need a bit of Windows 95

The times of Windows 95 are long gone. But some projects try to keep it alive. This project uses some system virtualization to bring you Windows 95 almost on any platform with almost no dependencies.

This is Windows 95, running in an Electron app. Yes, it’s the full thing. I’m sorry.

Felix Rieseberg

Miataru – open source location tracking

Not a lot of things are more private than your location.

Yet sometimes you wish to share your location with friends and family. May it be during an event or regularly. Maybe you want to

To allow the tech-minded audience to be in full control of what data is aggregated and stored regarding these needs I’ve created Miataru back in 2013 as an open-source project from end-2-end.

With the protocol being completely open and ready to be integrated into any home automation interested users can either utilize the publicly available (stores-nothing-on-disk) server or host your own.

Everything from the server to the clients is available in source and there’s a ready-to-go version of the client app on the AppStore.

this is a location sharing session when the blue pin met the yellow pin

OpenSource drawing: Krita

I just recently learned about Krita. An open source drawing application that allows you to… oh well… do free-hand drawings.

Krita is a FREE and open source painting tool designed for concept artists, illustrators, matte and texture artists, and the VFX industry. Krita has been in development for over 10 years and has had an explosion in growth recently. It offers many common and innovative features to help the amateur and professional alike. See below for some of the highlighted features.

Krita highlights

Taking a look at the gallery yields that I cannot draw. Frustration about that is limited because there’s so much nice drawings to gaze at!

Also this is a multi-platform application. It’s available for Windows, macOS and for Linux.

pushing your myfitnesspal data to MQTT

MyFitnessPal is a great online service we are using to track what we eat. It’s well integrated into our daily routine – it works!

Unfortunately MyFitnessPal is not well set-up to interface 3rd party applications with it. In fact it appears they are actively trying to make it harder for externals to utilize the data there.

To access your data there’s an open source project called “python-myfitnesspal” which allows you to interface with MyFitnessPal from the command line. This project uses web-scraping to extract the information from the website and will break everytime MyFitnessPal is changing the design/layout.

Since the output for this would be command line text output it is not of great use for a standardized system. What is needed is to have the data sent in a re-useable way into the automation system.

This is why I wrote the additional tool “myfitnesspal2mqtt“. It takes the output provided by python-myfitnesspal and sends it to an MQTT topic. The message then can be decoded, for example with NodeRed, and further processed.

As you can see in the image above I am taking the MQTT message coming from myfitnesspal2mqtt and decoding it with a bit of javascript and outputting it back to MQTT.

var complete = {};
var sodium = {};
var carbohydrates = {};
var calories = {};
var daydate = {};
var fat = {};
var sugar = {};
var protein = {};

var weight = {};
var bodyfat = {};


var goalsodium = {};
var goalcarbohydrates = {};
var goalcalories = {};
var goalfat = {};
var goalsugar = {};
var goalprotein = {};

var caloriesdiff = {};

var ttopic = msg.topic.toLowerCase();

var firstobject = Object.keys(msg.payload)[0];

complete.payload = msg.payload[firstobject].complete;
complete.topic = ttopic+'/complete';

sodium.payload = msg.payload[firstobject].totals.sodium;
sodium.topic = ttopic+'/total/sodium';
carbohydrates.payload = msg.payload[firstobject].totals.carbohydrates;
carbohydrates.topic = ttopic+'/total/carbohydrates';
calories.payload = msg.payload[firstobject].totals.calories;
calories.topic = ttopic+'/total/calories';
fat.payload = msg.payload[firstobject].totals.fat;
fat.topic = ttopic+'/total/fat';
sugar.payload = msg.payload[firstobject].totals.sugar;
sugar.topic = ttopic+'/total/sugar';
protein.payload = msg.payload[firstobject].totals.protein;
protein.topic = ttopic+'/total/protein';

weight.payload = msg.payload[firstobject].measurements.weight;
weight.topic = ttopic+'/measurement/weight';
bodyfat.payload = msg.payload[firstobject].measurements.bodyfat;
bodyfat.topic = ttopic+'/measurement/bodyfat';

goalsodium.payload = msg.payload[firstobject].goals.sodium;
goalsodium.topic = ttopic+'/goal/sodium';
goalcarbohydrates.payload = msg.payload[firstobject].goals.carbohydrates;
goalcarbohydrates.topic = ttopic+'/goal/carbohydrates';
goalcalories.payload = msg.payload[firstobject].goals.calories;
goalcalories.topic = ttopic+'/goal/calories';
goalfat.payload = msg.payload[firstobject].goals.fat;
goalfat.topic = ttopic+'/goal/fat';
goalsugar.payload = msg.payload[firstobject].goals.sugar;
goalsugar.topic = ttopic+'/goal/sugar';
goalprotein.payload = msg.payload[firstobject].goals.protein;
goalprotein.topic = ttopic+'/goal/protein';

caloriesdiff.payload = msg.payload[firstobject].goals.calories - msg.payload[firstobject].totals.calories;
caloriesdiff.topic = ttopic+'/caloriedeficit';

daydate.payload = firstobject;
daydate.topic = ttopic+"/date";

return [complete, sodium, carbohydrates, calories, fat, sugar, protein, weight, bodyfat, goalsodium, goalcarbohydrates, goalcalories, goalfat, goalsugar, goalprotein, daydate, caloriesdiff];

In the end it expands into a multitude of topics with one piece of information per MQTT topic.

And with just that every time the script is run (which I do in a docker container and with a cronjob) the whole lot of pieces of information about nutrition and health stats are being pushed and stored in the home automation system.

This way they are of course also available to the home automation system to do things with it.

Like locking the fridge.