stylizing video by example

AI and deep-learning is not always necessary or helpful. In this case impressive results have been achieved without the use of any of the hyped technologies.

In this case you give the algorithms two inputs. A video base that you want to stylize and a base picture that resembles the style you want to achieve.

We introduce a new example-based approach to video stylization, with a focus on preserving the visual quality of the style, user controllability and applicability to arbitrary video. Our method gets as input one or more keyframes that the artist chooses to stylize with standard painting tools. It then automatically propagates the stylization to the rest of the sequence. To facilitate this while preserving visual quality, we developed a new type of guidance for state-of-art patch-based synthesis, that can be applied to any type of video content and does not require any additional information besides the video itself and a user-specified mask of the region to be stylized. We further show a temporal blending approach for interpolating style between keyframes that preserves texture coherence, contrast and high frequency details. We evaluate our method on various scenes from real production setting and provide a thorough comparison with prior art.

Stylizing Video by Example Paper

Apparently there also is a Windows demo available in which you are supposedly be able to create your own stylized short clips. But as I wanted to try it out it threw a lot of funky messages regarding the application to be specifically untrustworthy / possibly malicious. So be aware and cautious.

color assimilation grid illusion

The image [..] is a visual/artistic experiment playing with simultanous contrast resulting from other experiments these days. An over-saturated colored grid overlayed on a  grayscale image causes the grayscale cells to be perceived as having color.
The processing needed to create the above image happened along with unrelated but significant code improvements In the last couple of weeks. I have been visiting mitch – a prolific GIMP contributors for collaboration – and lots of progress has been – and is still – being made on babl, GEGL and GIMP.

original article
See the effect in a video

You can now play with this effect and experiment for yourself. Get Gimp and enjoy!

electronic fireworks

The firecracker exploded. Apparently after 2 weeks of usage of the Chuwi Hi10 Air the eMMC flash is malfunctioning.

In a totally strange way: Every byte on the eMMC can be read, seemingly. Even Windows 10 boots. But after a while it will hang and blue screen. Apparently because it tries to write to the eMMC and when those writes fail and pile up in the caches at some point the system calls it quits.

Anyhow: It means that no byte that is right now on this eMMC can be deleted / overwritten but only be read.

The great chinese support is really helpful and offered to replace the device free of charge right away. That’s very nice! But I came to the conclusion that I cannot send the device in, because:

It contains a full set of synched private data that I cannot remove by all means because the freaking soldered-on eMMC flash is broken.

The recipient of this broken tablet in china would be able to read all my data and I could not do anything about it.

Only an extremely small fraction of data is on there unencrypted. Only that much I hadn’t yet switched on encryption on during the initial set-up I was still doing on the device. And that little piece of data already is what won’t let me send out the device.

Now, what can we learn from this? We can learn: Never ever ever work with anything, even during set-up, without full encryption.

visualize baby sleep pattern

A visualization of my son’s sleep pattern from birth to his first birthday. Crochet border surrounding a double knit body. Each row represents a single day. Each stitch represents 6 minutes of time spent awake or asleep

Seung Lee on Twitter

No babies here. But I want such a blanket now.

augmented (reality) book cover

If you ever want to quickly explain what augmented reality could be to a person not knowing yet, you might want to use this (and other) use cases for a visual explanation:

I achieved this by separating the artwork and text into many individual layers, that I placed in receding layers of 3D depth, in a 3D program on the computer. And made sure everything outside the borders of the book is excluded, to give it the ‘portal’ effect.

Augmented Reality Book Cover by Alexander Wand

full website screenshots from your commandline

Think of this: You want to capture a whole, multi-scroll-pages, web-page into one image.

This can be difficult without the right tools. It surely will be a lot of work to accomplish a 10th of thousand pixel height screenshot put together from multiple single screenshots…

CutyCapt is there to help! It’s a command line tool encapsulating the very powerful WebKit browser engine to render a full page and then create a single file screenshot of the whole page for you.

By example, this is what it did when told to screenshot this website:

PixelFed – Federated Image Sharing

In search of alternatives to the traditional centralized hosted social networks a lot of smart people have started to put time and thought into what is called “the-federation”.

The Federation refers to a global social network composed of nodes that talk to each other. Each of them is an installation of software which supports one of the federated social web protocols.

What is The Federation?

You may already have heard about projects like Mastodon, Diaspora*, Matrix (Synapse) and others.

The PixelFed project seems to gain some traction as apparently the first documentation and sources are made available.

PixelFed is a federated social image sharing platform, similar to instagram. Federation is done using the ActivityPub protocol, which is used by Mastodon, PeerTube, Pleroma, and more. Through ActivityPub PixelFed can share and interact with these platforms, as well as other instances of PixelFed.

the-federation

I am posting this here as to my personal logbook.

Given that there’s already a Dockerfile I will give it a try as soon as possible.

CCCamp 2019 – 21. – 25. August 2019

The Chaos Communication Camp is an international, five-day open-air event for hackers and associated life-forms. It provides a relaxed atmosphere for free exchange of technical, social, and political ideas. The Camp has everything you need: power, internet, food and fun. Bring your tent and participate!

CCCamp 2019 Wiki

It has been 2005 that I had the time and chance to attend an international open-air meeting of normal people. Of course I am talking about the 2005 What-the-hack I wrote about back then.

This year it’s time again for the Chaos Communication Camp in Germany. Sadly still I won’t be attending. Clearly that needs to change with one of the next iterations. With the CCC events becoming highly valuable also for families maybe it’s a chance in the future to meet up with old and valued friends (wink-wink Andreas Heil).

The Chaos Communication Camp (also known as CCCamp) is an international meeting of hackers that takes place every four years, organized by the Chaos Computer Club (CCC). So far all CCCamps have been held near Berlin, Germany.

The camp is an event for providing information about technical and societal issues, such as privacy, freedom of information and data security. Hosted speeches are held in big tents and conducted in English as well as German. Each participant may pitch a tent and connect to a fast internet connection and power.

CCCamp in Wikipedia

Enjoy the intro-movie that has just been made available to us, alongside the whole design material:

retrofuturism sketches

On twitter and artstation I’ve came across Sheng Lam. An artist with a quite fascinating way of sketching. Take a look and be inspired:

Concept artist with AAA experience. Currently working on Star Citizen at Cloud Imperium Games. Mostly draws and likes to eat delicious food.

Sheng Lam on ArtStation

a red triangle on the window

When you walk around in Tokyo you will find that many buildings have red-triangle markings on some of the windows / panels on the outside.

some of the windows have red triangles pointing down
do you see the triangles pointing down on the upper right wall?

I noticed them as well but I could not think of an explanation. Digging for information brought up this:

Panels to fire access openings shall be indicated with either a red or orange triangle of equal sides (minimum 150mm on each side), which can be upright or inverted, on the external side of the wall and with the wordings “Firefighting Access – Do Not Obstruct” of at least 25mm height on the internal side.

Singapore Firefighting Guide 2018

The red triangles on the buildings/hotel windows in Japan are the rescue paths to be used in case of fire. All fire fighters know the meaning of this red triangle on the windows. Red in color makes it prominent, to be located easily by the fire fighters in case of a fire incident. During a fire incident, windows are generally broken to allow for smoke and other gases to come out of the building.

Triangles in Japan

グランツリー武蔵小杉 and Park City Forest Towers on canvas

I’ve blogged about those pictures taken day- and night-time before. I’ve also blogged about how they where produced.

With a picture spot freed at one of the walls in our house we decided to print the day and night pictures on canvas and have them side-by-side. And it looks great, I think.

I might want to take the night picture again with the slightly different perspective
close up on the canvas structure

Bitmap & tilemap generation with the help of ideas from quantum mechanics

You can get a grasp at the beautiful side of science with visualizations and algorithms that output visual results.

This is the example of producing lots and lots of complex data (houses!) from a small set of input data. It is widely used in game development but also can be helpful to generate parameterized test and simulation environments for machine learning.

So before sending you over to the more detailed explanation the visual example:

This is a lot of different house images. Those are generated using a program called WaveFunctionCollapse:

WFC initializes output bitmap in a completely unobserved state, where each pixel value is in superposition of colors of the input bitmap (so if the input was black & white then the unobserved states are shown in different shades of grey). The coefficients in these superpositions are real numbers, not complex numbers, so it doesn’t do the actual quantum mechanics, but it was inspired by QM. Then the program goes into the observation-propagation cycle:

On each observation step an NxN region is chosen among the unobserved which has the lowest Shannon entropy. This region’s state then collapses into a definite state according to its coefficients and the distribution of NxN patterns in the input.

On each propagation step new information gained from the collapse on the previous step propagates through the output.

On each step the overall entropy decreases and in the end we have a completely observed state, the wave function has collapsed.

It may happen that during propagation all the coefficients for a certain pixel become zero. That means that the algorithm has run into a contradiction and can not continue. The problem of determining whether a certain bitmap allows other nontrivial bitmaps satisfying condition (C1) is NP-hard, so it’s impossible to create a fast solution that always finishes. In practice, however, the algorithm runs into contradictions surprisingly rarely.

Wave Function Collapse algorithm has been implemented in C++PythonKotlinRustJuliaGoHaxeJavaScript and adapted to Unity. You can download official executables from itch.io or run it in the browser. WFC generates levels in Bad NorthCaves of Qudseveral smaller games and many prototypes. It led to new research. For more related workexplanationsinteractive demosguidestutorials and examples see the ports, forks and spinoffs section.

Tilt your head, be more aggressive

“We show that tilting one’s head downward systematically changes the way the face is perceived, such that a neutral face—a face with no muscle movement or facial expression—appears to be more dominant when the head is tilted down,” explain researchers Zachary Witkower and Jessica Tracy of the University of British Columbia. 

A Facial-Action Imposter

Look me in the eye

As tools get better artists really polish out what technology enables them to do with computers nowadays.

This eye above made me awww. It’s from a user “ChrisJones” in the Blender Forum who posted his progress on modelling a human head in upmost detail.

Go there and be awwd as well. And while you’re there grab a copy of Blender and give it a go.

絵描きさんの作業環境が見たい – I want to see artists’ work environments

We are using computers every day and we are doing this in many different environments. Some of us give their desk space and work environment some more thoughts.

If you want some inspiration regarding your desk and work space, take a look at this great Twitter Hashtag: #絵描きさんの作業環境が見たい

It means “I want to see artists’ work environments” and is used for some years now for japanese artists to post pictures of their work environments…

I also had posted mine, yet not being an artist.

N8.5 – Tokyu White

It always seems to me so much easier to meet a Hackers in Japan than anywhere else. Maybe social-gravity works different there. Or there are more of them.

Whatever the reason, I keep crossing paths with people that more or less deep-dive into topics and understand systems in ways and in details that you would normally not expect.

And todays example is a detail you would normally not think about:

The footbridge that was newly built in Shibuya was painted in N8.5. This is commonly known as “Tokyu White”…

はらしょ‏ on Twitter

“Tokyu White”, N8.5, … of course…if you would need a bucket of this you can order it online.

technical visualization tools

There’s so much happening in this field as visualizations become more powerful and easier to create.

WaveDrom

WaveDrom draws your Timing Diagram or Waveform from simple textual description. It comes with description language, rendering engine and the editor.

WaveDrom editor works in the browser or can be installed on your system. Rendering engine can be embeded into any webpage.

https://wavedrom.com/

MermaidJS

Generation of diagrams and flowcharts from text in a similar manner as markdown.

Ever wanted to simplify documentation and avoid heavy tools like Visio when explaining your code?

This is why mermaid was born, a simple markdown-like script language for generating charts from text via javascript. Try it using our editor.

DLG Gütesiegel in japanese TV commercials

Catching up to some YouTube things during dinner made us see a Japanese TV commercial sporting the German DLG quality seal. Funny to see that these kind of quality seals carry over even to Japan…

The German Agricultural Society (Deutsche Landwirtschafts-Gesellschaft), commonly known as DLG, is an international non-profit organisation for agricultural industry in Germany. DLG was founded in 1885 by Max Eyth, has over 23,000 members as of 2011 and is headquartered in Frankfurt am Main. Its main purpose is to promote technical progress and scientific advances in the food and agricultural industry, including setting standards.

DLG

look at the earth

Watch sunlight and weather patterns move across Earth throughout the day, and bask in the glory of our blue marble in real time.

Every 20 minutes (or every hour, you pick), Downlink updates your desktop background with the newest images of Earth.

Choose from 8 different views of Earth, including stunning full disk images from 3 different geostationary satellites.

Downlink is a macOS application which downloads the most current image taken by earth orbitting satellites. So what you see is as accurately depicting reality as technically possible right now.

Interestingly all this data comes from public domain sources. And the makers of the app have documented their sources:

Thanks to NASA, NOAA, JMA, Lockheed Martin, Harris Corporation, ULA, MHI, and everyone else who designed, built, launched, and operate GOES-16, GOES-17, and Himawari-8.
If you’d like to use the sources Downlink is built on top of, here is the file pointing to those resources. Use it, and tell me what you’ve built!

I am using macOS as well as Linux as my main desktop operating systems. My macs are set with Downlink. But what about my Linux machines?

Easy! The file referring to the sources looks like this:

With some simple steps on Linux we can grab the URL we want, download the image and update the desktop background.

Step 1: Getting the URL from this JSON.

Install jq, curl and xargs and in your shell run this:

curl -s http://downlinkapp.com/sources.json | jq -r '.sources[2].url.full' 

This will give you the URL “full” of the 2nd source in the list (Hiwamari-8).

Step 2: Download the image.

curl -s http://downlinkapp.com/sources.json | jq -r '.sources[2].url.full' | xargs -I{} curl -s -o background.jpg {}

By simply adding the xargs and curl call it will take the URL output from the command in front as input to the second curl call and download it. It also will store the file as “background.jpg”.

Step 3: Setting as background image.

Setting the background image of your desktop depends on what desktop software you are using on Linux. Depending on that you will need to look up in the manual how to set the background image from the command line.

But for most use cases there’s a tool that helps. It’s called Variety.

Variety is an open-source wallpaper changer for Linux
Variety is packed with great features, yet slim and easy to use. It can use local images or automatically download wallpapers from Unsplash and other online sources, allows you to rotate them on a regular interval, and provides easy ways to separate the great images from the junk. Variety can also display wise and funny quotations or a nice digital clock on the desktop.

Earth, Moon, Spaceship.

As you might know humans are going to go “back to the moon“. At least when you ask a certain eCommerce mogul.

We have been there already. 40+ years ago. And since then we have unlearnt how to do it.

Just to give an impression of what we as humans where capable to achieve 40 years ago:

Moon (below), Apollo 16 Command Module (left), Earth (right)

There’s a whole collection of these photos freely available for further digging. Go there and spend a day!

Imagine: Humans flew there and back. With 60s tech. What can we do now when we want it?

Of course. The important bit is not really flying there and back. The most important thing for me is perspective.

Look at that small blue-white marble. We should reset our perspectives regularly.