“Mono is a software platform designed to allow developers to easily create cross platform applications. It is an open source implementation of Microsoft’s .Net Framework based on the ECMA standards for C# and the Common Language Runtime. We feel that by embracing a successful, standardized software platform, we can lower the barriers to producing great applications for Linux.” (Source)

In other words: Mono is the platform which is needed to run the sones GraphDB on any operating system different from Windows. It included the so called “Mono Runtime” which basically is the place where the sones GraphDB “lives” to do it’s work.

Being a runtime is not an easy task. In fact it’s abilities and algorithms take a deep impact on the performance of the application that runs on top of it. When it comes to all things related to memory management the garbage collector is one of the most important parts of the runtime:

“In computer science, garbage collection (GC) is a form of automatic memory management. It is a special case of resource management, in which the limited resource being managed is memory. The garbage collector, or just collector, attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program. Garbage collection was invented by John McCarthy around 1959 to solve problems in Lisp.” (Source)

The Mono runtime has always used a simple garbage collector implementation called “Boehm-Demers-Weiser conservative garbage collector”. This implementation is mainly known for its simplicity. But as more and more data intensive applications, like the sones GraphDB, started to appear this type of garbage collector wasn’t quite up to the job.

So the Mono team started the development on a Simple Generational Garbage collector whose properties are:

• Two generations.
• Mostly precise scanning (stacks and registers are scanned conservatively).
• Copying minor collector.
• Two major collectors: Copying and Mark&Sweep.
• Per-thread fragments for fast per-thread allocation.
• Uses write barriers to minimize the work done on minor collections.

To fully understand what this new garbage collector does you most probably need to read this and take a look inside the mono s-gen garbage collector code.

So what we did was taking the old and the new garbage collector and our GraphDB and let them iterate through an automated test which basically runs 200.000 insert queries which result in more than 3.4 million edges between more than 120.000 objects. The results were impressive when we compared the old mono garbage collector to the new mono-sgen garbage collector.

When we plotted a basic graph of the measurements we got that:

On the x-axis it’s the number of inserts and on the y-axis it’s the time it takes to answer one query. So it’s a great measurement to see how big actually the impact of the garbage collector is on a complex application like the sones GraphDB.

The red curve is the old Boehm-Demers-Weiser conservative garbage collector built into current stable versions of mono. The blue curve is the new SGEN garbage collector which can be used by invoking Mono using the “mono-sgen” command instead of the “mono” command. Since mono-sgen is not included in any stable build yet it’s necessary to build mono from source. We documented how to do that here.

So what are we actually seeing in the chart? We can see that mono-sgen draws a fairly linear line in comparison to the old mono garbage collector. It’s easy to tell why the blue curve is rising – it’s because the number of objects is growing with each millisecond. The blue line is just what we are expecting from a hard working garbage collector. To our surprise the old garbage collector seems to have problems to cope with the number of objects over time. It spikes several times and in the end it even gets worse by spiking all over the place. That’s what we don’t want to see happening anywhere.

The conclusion is that if you are running something that does more than printing out “Hello World” on Mono you surely want to take a look at the new mono-sgen garbage collector. If you’re planning to run the sones GraphDB on Mono we highly recommend to use mono-sgen.

It’s about time to import some data into our previously established object scheme. If you want to do this yourself you want to first run the Crunchbase mirroring tool and create your own mirror on your hard disk.

In the next step another small tool needs to be written. A tool that creates nice clean GQL import scripts for our data. Since every data source is different there’s not really a way around this step – in the end you’ll need to extract data here and import data here. One possible different solution could be to implement a dedicated importer for the GraphDB – but I’ll leave that for another article series. Back to our tool: It’s called “First-Import” and it’s only purpose is to create a first small graph out of the mirrored Crunchbase data and fill the mainly primitive data attributes. Download this tool here.

This is why in this first step we mainly focus on the following object types:

• Company
• FinancialOrganization
• Person
• Product
• ServiceProvider

Additionally all edges to a company object and the competition will be imported in this part of the article series.

So what does the first-import tool do? Simple:

1. it deserializes the JSON data into a useable object – in this case it’s written in C# and uses .NETs own JavaScript deserializer
2. it then maps all attributes of that deserialized JSON object to attribute names in our graph data object scheme and it does so by outputting a simple query
   1. Simple Attribute Types like String and Integer are just simply assigned using the “=” operator in the Graph Query Language
   2. 1:1 References are assigned by assigning a REF(…) to the attribute – for example: INSERT INTO Product VALUES (Company = REF(Permalink=’companyname’))
   3. 1:n References are assigned by assigning a SETOF(…) to the attribute – because we are not using a bulk import interface but the standard GQL REST Interface it’s necessary that the object(s) we’re going to reference are already in existence – therefore we chose to do this 1:n linking step after creating the objects itself in a separate UPDATE step. Knowing this the UPDATE looks like this: UPDATE Company SET (ADD TO Competitions SETOF(permalink=’…’,permalink=’…’)) WHERE Permalink = ’companyname’

For the most part of the work it’s copy-n-paste to get the first-import tool together – it could have been done in a more sophisticated way (like using reflection on the deserialized JSON objects) but that’s most probably part of another article.

When run in the “crunchbase” directory created by the Crunchbase Mirroring tool the first-import tool generates GQL scripts – 6 of them to be precise:

The last script is named “Step_3” because it’s supposed to come after all the others.

These scripts can be easily imported after establishing the object scheme. The thing is though – it won’t be that fast. Why is that? We’re creating several thousand nodes and the edges between them. To create such an edge the Query Language needs to identify the node the edge originates and the node the edge should point to. To find these nodes the user is free to specify matching criteria just like in a WHERE clause.

So if you do a UPDATE Company SET (ADD TO Competitions SETOF(Permalink=’company1’,Permalink=’company2’)) WHERE Permalink = ’companyname’ the GraphDB needs to access the node identified by the Permalink Attribute with the value “companyname” and the two nodes with the values “company1” and “company2” to create the two edges. It will work just like all the scripts are but it won’t be as fast as it could be. What can help to speed up things are indices. Indices are used by the GraphDB to identify and find specific objects. These indices are used mainly in the evaluation of a WHERE clause.

The sones GraphDB offers a number of integrated indices, one of which is HASHTABLE which we are going to use in this example. Furthermore everyone interested can implement it’s own index plugin – we will have a tutorial how to do that online in the future – if you’re interested now just ask how we can help you to make it happen!

Back to the indices in our example:

The syntax of creating an index is quite easy, the only thing you have to do is tell the CREATE INDEX query on which type and attribute the index should be created and of which indextype the index should be. Since we’re using the Permalink attribute of the Crunchbase objects as an identifier in the example (it could be any other attribute or group of attributes that identify one particular object) we want to create indices on the Permalink attribute for the full speed-up. This would look like this:

• CREATE INDEX ON Company (Permalink) INDEXTYPE HashTable
• CREATE INDEX ON FinancialOrganization (Permalink) INDEXTYPE HashTable
• CREATE INDEX ON Person (Permalink) INDEXTYPE HashTable
• CREATE INDEX ON ServiceProvider (Permalink) INDEXTYPE HashTable
• CREATE INDEX ON Product (Permalink) INDEXTYPE HashTable

Looks easy, is easy! To take advantage of course this index creation should be done before creating the first nodes and edges.

After we got that sorted the only thing that’s left is to run the scripts. This will, depending on your machine, take a minute or two.

So after running those scripts what happened is: all Company, FinancialOrganization, Person, ServiceProvider and Product objects are created and filled with primitive data types

1. all attributes which are essentially references (1:1 or 1:n) to a Company object are being set, these are
   1. Company.Competitions
   2. Product.Company

That’s it for this part – in the next part of the series we will dive deeper into connecting nodes with edges. There is a ton of things that can be done with the data – stay tuned for the next part.

After the overview and the first use-case introduction it’s about time to play with some data objects.

So how can one actually access the data of crunchbase? Easy as pie: Crunchbase offers an easy to use interface to get all information out of their database in a fairly structured JSON format. So what we did is to write a tool that actually downloads all the available data to a local machine so we can play with it as we like in the following steps.

This small tool is called MirrorCrunchbase and can be downloaded in binary and sourcecode here. As for all sourcecode and tools in this series this runs on windows and linux (mono). You can use the sourcecode to get an impression what’s going on there or just the included binaries (in bin/Debug) to mirror the data of Crunchbase.

To say a few words about what the MirrorCrunchbase tool actually does first a small source code excerpt:

So first it gets the list of all objects like the company names and then it retrieves each company object according to it’s name and stores everything in .js files. Easy eh?

When it’s running you get an output similar to that:

And after the successful completion you should end up with a directory structure

The .js files store basically every information according to the data scheme overview picture of part 2.  So what we want to do now is to transform this overview into a GQL data scheme we can start to work with. A main concept of sones GraphDB is to allow the user to evolve a data scheme over time. That way the user does not have to have the final data scheme before the first create statement. Instead the user can start with a basic data scheme representing only standard data types and add complex user defined types as migration goes along. That’s a fundamentally different approach from what database administrators and users are used to today.

Todays user generated data evolves and grows and it’s not possible to foresee in which way attributes need to be added, removed, renamed. Maybe the scheme changes completely. Everytime the necessity emerged to change anything on a established and populated data scheme it was about time to start a complex and costly migration process. To substantially reduce or even in some cases eliminate the need for such a complex process is a design goal of the sones GraphDB.

In the Crunchbase use-case this results in a fairly straight-forward process to establish and fill the data scheme. First we create all types with their correct name and add only those attributes which can be filled from the start – like primitives or direct references. All Lists and Sets of Edges can be added later on.

So these would be the Create-Type Statements to start with in this use-case:

• CREATE TYPE Company ATTRIBUTES ( String Alias_List, String BlogFeedURL,    String BlogURL, String Category, DateTime Created_At, String CrunchbaseURL, DateTime Deadpooled_At, String Description, String EMailAdress, DateTime Founded_At, String HomepageURL, Integer NumberOfEmployees, String Overview, String Permalink, String PhoneNumber, String Tags, String TwitterUsername, DateTime Updated_At, Set<Company> Competitions )

• CREATE TYPE FinancialOrganization ATTRIBUTES ( String Alias_List, String BlogFeedURL, String BlogURL, DateTime Created_At, String CrunchbaseURL, String Description, String EMailAdress, DateTime Founded_At, String HomepageURL, String Name, Integer NumberOfEmployees, String Overview, String Permalink, String PhoneNumber, String Tags, String TwitterUsername, DateTime Updated_At )

• CREATE TYPE Product ATTRIBUTES ( String BlogFeedURL, String BlogURL, Company Company, DateTime Created_At, String CrunchbaseURL, DateTime Deadpooled_At, String HomepageURL, String InviteShareURL, DateTime Launched_At, String Name, String Overview, String Permalink, String StageCode, String Tags, String TwitterUsername, DateTime Updated_At)

• CREATE TYPE ExternalLink ATTRIBUTES ( String ExternalURL, String Title )

• CREATE TYPE EmbeddedVideo ATTRIBUTES ( String Description, String EmbedCode )

• CREATE TYPE Image ATTRIBUTES ( String Attribution, Integer SizeX, Integer SizeY, String ImageURL )

• CREATE TYPE IPO ATTRIBUTES ( DateTime Published_At, String StockSymbol, Double Valuation, String ValuationCurrency )

• CREATE TYPE Acquisition ATTRIBUTES ( DateTime Acquired_At, Company Company, Double Price, String PriceCurrency, String SourceDestination, String SourceURL, String TermCode )

• CREATE TYPE Office ATTRIBUTES ( String Address1, String Address2, String City, String CountryCode, String Description, Double Latitude, Double Longitude, String StateCode, String ZipCode )

• CREATE TYPE Milestone ATTRIBUTES ( String Description, String SourceDescription, String SourceURL, DateTime Stoned_At )

• CREATE TYPE Fund ATTRIBUTES ( DateTime Funded_At, String Name, Double RaisedAmount, String RaisedCurrencyCode, String SourceDescription, String SourceURL )

• CREATE TYPE Person ATTRIBUTES ( String AffiliationName, String Alias_List, String Birthplace, String BlogFeedURL, String BlogURL, DateTime Birthday, DateTime Created_At, String CrunchbaseURL, String FirstName, String HomepageURL, Image Image, String LastName, String Overview, String Permalink, String Tags, String TwitterUsername, DateTime Updated_At )

• CREATE TYPE Degree ATTRIBUTES ( String DegreeType, DateTime Graduated_At, String Institution, String Subject )

• CREATE TYPE Relationship ATTRIBUTES ( Boolean Is_Past, Person Person, String Title )

• CREATE TYPE ServiceProvider ATTRIBUTES ( String Alias_List, DateTime Created_At, String CrunchbaseURL, String EMailAdress, String HomepageURL, Image Image, String Name, String Overview, String Permalink, String PhoneNumber, String Tags, DateTime Updated_At )

• CREATE TYPE Providership ATTRIBUTES ( Boolean Is_Past, ServiceProvider Provider, String Title )

• CREATE TYPE Investment ATTRIBUTES ( Company Company, FinancialOrganization FinancialOrganization, Person Person )

• CREATE TYPE FundingRound ATTRIBUTES ( Company Company, DateTime Funded_At, Double RaisedAmount, String RaisedCurrencyCode, String RoundCode, String SourceDescription, String SourceURL )

You can directly download the according GQL script here. If you use the sonesExample application from our open source distribution you can create a subfolder “scripts” in the binary directory and put the downloaded script file there. When you’re using the integrated WebShell, which is by default launched on port 9975 an can be accessed by browsing to http://localhost:9975/WebShell you can execute the script using the command “execdbscript” followed by the filename of the script.

As you can see it’s quite straight forward a copy-paste action from the graphical scheme. Even references are not represented by a difficult relational helper, instead if you want to reference a company object you can just do that (we actually did that – look for example at the last line of the gql script above). As a result when you execute the above script you get all the Types necessary to fill data in in the next step.

So that’s it for this part – in the next part of this series we will start the initial data import using a small tool which reads the mirrored data and outputs gql insert queries.

Where to start: existing data scheme and API

This series already tells in it’s name what the use case is: The “CrunchBase”.  On their website they speak for themselves to explain what it is: “CrunchBase is the free database of technology companies, people, and investors that anyone can edit.”. There are many reasons why this was chosen as a use-case. One important reason is that all data behind the CrunchBase service is licensed under Creative-Commons-Attribution (CC-BY) license. So it’s freely available data of high-tech companies, people and investors.

Currently there are more than 40.000 different companies, 51.000 different people and 4.200 different investors in the database. The flood of information is big and the scale of connectivity even bigger. The graph represented by the nodes could be even bigger than that but because of the limiting factors of current relational database technology it’s not feasible to try to do that.

sones GraphDB is coming to the rescue: because it’s optimized to handle huge datasets of strongly connected data. Since the CrunchBase data could be uses as a starting point to drive connectivity to even greater detail it’s a great use-case to show these migration and handling.

Thankfully the developers at CrunchBase already made one or two steps into an object oriented world by offering an API which answers queries in JSON format. By using this API everyone can access the complete data set in a very structured way. That’s both good and bad. Because the used technologies don’t offer a way to represent linked objects they had to use what we call “relational helpers”. For example: A person founded a company. (person and company being a JSON object). There’s no standardized way to model a relationship between those two. So what the CrunchBase developers did is they added an unique-Identifier to each object. And they added a new object which is uses as a “relational helper”-object. The only purpose of these helper objects is to point towards a unique-identifier of another object type. So in our example the relationship attribute of the person object is not pointing directly to a specific company or relationship, but it’s pointing to the helper object which stores the information which unique-identifier of which object type is meant by that link.

To visualize this here’s the data scheme behind the CrunchBase (+all currently available links):

As you can see there are many more “relational helper” dead-ends in the scheme. What an application had to do up until now is to resolve these dead-ends by going the extra mile. So instead of retrieving a person and all relationships, and with them all data that one would expect, the application has to split the data into many queries to internally build a structure which essentially is a graph.

Another example would be the company object. Like the name implies all data of a company is stored there. It holds an attribute called investments which isn’t a primitive data type (like a number or text) but a user defined complex data type. This user defined data type is called List<FundingRoundStructure>. So it’s a simple list of FundingRoundStructure objects.

When we take a look at the FundingRoundStructure there’s an attribute called company which is made up by the user defined data type CompanyStructure. This CompanyStructure is one of these dead-ends because there’s just a name and a unique-id. The application now needs retrieve the right company object with this unique-id to access the company information.

Simple things told in a simple way: No matter where you start, you always will end up in a dead-end which will force you to start over with the information you found in that dead-end. It’s not user-friendly nor easy to implement.

The good news is that there is a way to handle this type of data and links between data in a very easy way. The sones GraphDB provides a rich set of features to make the life of developers and users easier. In that context: If we would like to know which companies also received funding from the same investor like let’s say the company “facebook” the only thing necessary would be one short query. Beside that those “relational helpers” are redundant information. That means in a graph database this information would be stored in the form of edges but not in any helper objects.

The reason why the developers of CrunchBase had to use these helpers is that JSON and the relational table behind it isn’t able to directly store this information or to query it directly. To learn more about those relational tables and databases try this link.

I want to end this part of the series with a picture of the above relational diagram (without the arrows and connections).

The next part of the series will show how we can access the available information and how a graph scheme starts to evolve.

If you want to explain how easy it is for a user or developer to use the sones GraphDB to work on existing datasets you do that by showing him an example – a use case. And this is exactly what this short series of articles will do: It’ll show the important steps and concepts, technologies and designs behind the use case and the sones GraphDB.

The sones GraphDB is a DBMS focusing on strong connected unstructured and semi-structured data. As the name implies these data sets are organized in Nodes and Edges objectoriented in a graph data structure.

“a simple graph”

To handle these complex graph data structures the user is given a powerful toolset: the graph query language. It’s a lot like SQL when it comes to comprehensibility – but when it comes to functionality it’s completely designed to help the user do previously tricky or impossible things with one easy query.

This articles series is going to show how real conventional-relational data is aggregated and ported to an easy to understand and more flexible graph datastructure using the sones GraphDB. And because this is not only about telling but also about doing we will release all necessary tools and source codes along with this article. That means: This is a workshop and a use case in one awesome article series.

The requirements to follow all steps of this series are: You want to have a working sone GraphDB. Because we just released the OpenSource Edition Version 1.1 you should be fine following the documentation on how to download and install it here. Beside that you won’t need programming skills but if you got them you can dive deep into every aspect. Be our guest!

This first article is titled “Overview” and that’s what you’ll get:

part 1: Overview

part 2: A short introduction into the use-case and it’s relational data

part 3: Which data and how does a GQL data scheme start?

part 4: The initial data import

part 5:  Linking nodes and edges: What’s connected with what and how does the scheme evolve?

part 6: Querying the data and how to access it from applications?

Well if you want just the essence of information that makes you go faster on your daily tasks cheat sheets are just that: the essence of information.

Today I found this cheat sheet particularly useful:

Source: http://zrusin.blogspot.com/2007/09/git-cheat-sheet.html

Every once in a while you download some code and fire up your Visual Studio and find out that this particular solution was once associated to a team foundation server you don’t know or have a login to. Like when you download source code from CodePlex and you get this “Please type in your username+password for this CodePlex Team Foundation Server”.

Or maybe you’re working on your companies team foundation server and you want to put some code out in the public. You surely want to get rid of these Team Foundation Server bindings.

There’s a fairly complicated way in Visual Studio to do this but since I was able to produce unforseen side effects I do not recommend it.

So what I did was looking into those files a Visual Studio Solution and Project consists of. And I found that there are really just a few files that hold those association information. As you can see in the picture below there are several files side by side to the .sln and .csproj files – like that .vssscc and .vspscc file. Even inside the .csproj and .sln file there are hints that lead to the team foundation server – so obviously besides removing some files a tool would have to edit some files to remove the tfs association.

So I wrote such a tool and I am going release it’s source code just beneath this article. Have fun with it. It compiles with Visual Studio and even Mono Xbuild – actually I wrote it with Monodevelop on Linux Multi-platform galore! Who would have thought of that in the founding days of the .NET platform?

So this is easy – this small tool runs on command line and takes one parameter. This parameter is the path to a folder you want to traverse and remove all team foundation server associations in. So normally I take a check-out folder and run the tool on that folder and all its subfolders to remove all associations.

So if you want to have this cool tool you just have to click here: Sourcecode Download

The effort of 10 days materializes in a Microsoft Surface demo. And you can see it at MSDN Developer Kino every day during CeBIT.

At sones I am involved in a project that works with a piece of hardware I wanted to work with for about 3 years now: the Microsoft Surface Table.

I was able to play with some tables every now and then but I never had a “business case” which contained a Surface. Now that case just came to us: sones is at the CeBIT fair this year – we were invited by Microsoft Germany to join them and present our cool technology along with theirs.

Since we already had a graph visualisation tool the idea was to bring that tool to Surface and use the platform specific touch controls and gestures.

the VisualGraph application that gave the initial idea

The good news was that it’s easier than thought to develop an application for Surface and all parties are highly committed to the project. The bad news is that we were short on time right from the start: less than 10 days from concept to live presentation isn’t the definition of “comfortable time schedule”. And since we’re currently in the process of development it’s a continueing race.

Thankfully Microsoft is committed to a degree they even made it possible to have two great Surface and WPF ninjas who enable is to get up to speed with the project (thanks to Frank Fischer, Andrea Kohlbauer-Hug, Rainer Nasch and Denis Bauer, you guys rock!).

a Surface simulator

I was able to convice UID to jump in and contribute their designing and user interface knowledge to our little project (thanks to Franz Koller and Cristian Acevedo).

During the process of development I made some pictures which will be used here and there promoting the demonstration. To give you an idea of the progress we made here’s a before and after picture:

We started with a simple port of VisualGraph to the surface table…

…and had something better working and looking at the end of that day.

I think everyone did a great job so far and will continue to do so – a lot work to be done till CeBIT!

Source 1: http://www.sones.com
Source 2: http://www.microsoft.de
Source 3: http://www.uid.com/

We want to show you something today: Not everybody has an idea what to think and do with a graph data structure. Not even talking about a whole graph database management system. In fact what everybody needs is something to get “in touch” with those kinds of data representations.

To make the graphs you are creating with the sones GraphDB that much more touchable we give you a sneak peak at our newest addition of the sone GraphDB toolset: the VisualGraph tool.

This tool connects to a running database and allows you to run queries on that database. The result of those queries is then presented to you in a much more natural and intuitive way, compared to the usual JSON and XML outputs. Even more: you can play with your queries and your data and see and feel what it’s like to work with a graph.

Expect this tool to be released in the next 1-2 months as open source. Everyone can use it, Everyone can benefit from it.

Oh. Almost forgot the video:

(Watch it in full screen if you can)

As you may know, my team and I are developing a graph database. A graph database is a database which is able to handle such things as the following:

social graph

So instead of tables with rows and columns, a graph database concentrates on objects and the connections between them and is therefore forming a graph which can be queried, traversed, whatever-you-might-want-to-do.

Lately more and more companies start realizing that their demand for storing unstructured data is growing. Reflecting on unstructured data, I always think of data which cannot single-handedly be mapped in columns and rows (e.g. tables). Normally complex relations between data are represented in relation-tables only containing this relational information. The complexity to query these data structures is humongous as the table based database needs to ‘calculate’ (JOINs, …) the relations every time they are queried. Even though modern databases cache these calculations the costs in terms of memory and cpu time are huge.

Graph databases more or less try to represent this graph of objects and edges (as the relations are called there) as native as possible. The sones GraphDB we have been working on for the last 5 years does exactly that: It stores and queries a data structure which represents a graph of objects. Our approach is to give the user a simple and easy to learn query language and handle all the object storage and object management tasks in a fully blown object oriented graph database developed from the scratch.

Since not everybody seems to have heard of graph databases, we thought it might be a good idea to lower barriers by providing personalized test instances. Everyone can get one of these without the need to install anything – a working AJAX/Javascript compatible browser will suit all needs. (get your instance here.)

Of course the user can choose between different ways to access the database test instance (like SOAP and REST) but the one we just released only needs a browser.

The sones GraphDB WebShell – as we call it – resembles a command line interface. The user can type a query and it is instantly executed on the database server and the results are presented in either a xml, json or text format.

Granted – the interested user needs to know about the query language and the possible usage scenarios. Everyone can access a long and a short documentation here.

Source 1: http://en.wikipedia.org/wiki/Social_graph
Source 2: http://www.sones.com
Source 3: Long documentation
Source 4: Short documentation

Since we are developers we do need tools to note and draw what we think would solve the problems of this planet.

One way to draw a sequence of actions would be a sequence diagram. There are a nbumber of tools to draw them but now I came across a web service that would allow me to write my sequence diagram in a easy textual representation and then it draws the diagram for me. Great stuff!

Source 1: http://en.wikipedia.org/wiki/Sequence_diagram
Source 2: http://websequencediagrams.com/

… you can use the “__MonoCS_” pre-processor flag.

I am managing my appointments using Outlook on windows and iCal on OS X. Since I am not using any Exchange service right now I was happy to find out that Outlook offers a functionality to export a local calendar automatically to an iCalendar compatible ICS file. Great feature but it lacks some things I desperately need.

Since I am managing my private and my business appointments in the same calendar, differentiating just by categories, I had a hard time configuring outlook to export a) an ics file containing all business appointments and b) an ics file containing all private appointments. It’s not possible to make the story short.

So I fired up Visual Studio as usual and wrote my own filter tool. I shall call it “iCalFilter”. It’s name is as simple as it’s functionality and code. I am releasing it under BSD license including the sources so everyone can use and modify it.

It’s a command line tool which should compile on Microsoft .NET and Mono. It takes several command line parameters like:

1. Input-File
2. Output-File
3. “include” or “exclude” –> this determines if the following categories are included or excluded in the output file
4. a list of categories separated by spaces
5. an optional parameter “-remove-description” which, if entered, removes all descriptions from events and alarms

Easy, eh?!

Grab the Source and Binary here: http://dropbox.schrankmonster.de/dropped/iCalFilter01.zip

Was für ein Tag. Nachdem wir vor ein paar Tagen nach viel harter Arbeit die “Technical Preview” unseres Babys “graphDB” gestartet haben hat nun auch der heise Verlag – namentlich die iX die frohe Kunde aufgegriffen und einen entsprechenden Artikel im Newsticker veröffentlich.

Wenn man sich auf jede Instanz die im Moment für Tester läuft ein Login geben lässt sieht das übrigends so aus:

Wundervoll zu sehen dass die Arbeit von exzellenten Entwicklern entsprechende Würdigung durch Kunden erhält. Interesse ist gut und ich denke in Zukunft wird man noch viel von der sones graphDB hören!

Source: http://www.heise.de/newsticker/meldung/Objektorientierte-Datenbank-als-Webservice-866041.html

I am proud to anounce that there’s a video publicly available which shows parts and projects Microsoft Research is working on currently. It’s great to see theses projects, concepts and ideas become publicly available one by one:

“Craig Mundie, chief research and strategy officer of Microsoft, presents “Rethinking Computing,” a look a how software and information technology can help solve the most pressing global challenges we face today. Part of UW’s Computer Science and Engineering’s Distinguished Lecture Series, Mundie demonstrates a number of current and future-looking technologies that show how computer science is changing scientific exploration and discovery in exciting ways. He discusses the role of new science in solving the global energy crisis, and answer questions from the audience.”

Source: http://www.uwtv.org/programs/displayevent.aspx?rID=30363&fID=6021

Great stuff ahead – this is just the thing I would want to write if it’s not been written already. This tool is free and open source and it’s the perfect workaround for those usual cases when you want to download a podcast in your holiday and your apple branded device tells you “You can only download files up to 10 Megabyte over 3G connections” – You take your notebook, log into 3G, create a WiFi Hotspot with this tool and off you go.

“Over the last week some of you may have heard about Connectify. It’s an app that unleashes the “Virtual WiFi” and Wireless Hosted Network features of Windows 7 to turn a PC into a Wireless Access Point or Hot Spot. Well, I looked into what it would take to build such an app, and it really wasn’t that difficult since Windows 7 has all the API’s built in to do it. After some time of looking things up and referencing the “Wireless Hosted Network” C++ sample within the WIndows 7 SDK, I now have a nice working version of the application to release. I’m calling this project “Virtual Router” since it essentially allows you to host a software based wireless router from your laptop or other PC with a Wifi card. Oh, and did I mention that this is FREE and OPEN SOURCE!”

“The Wireless Network create/shared with Virtual Router uses WPA2 Encryption, and there is not way to turn off that encryption. This is actually a feature of the Wireless Hosted Network API’s built into Windows 7 and 2008 R2 to ensure the best security possible.
You can give your "virtual" wireless network any name you want, and also set the password to anything. Just make sure the password is at least 8 characters.”

Source: http://virtualrouter.codeplex.com/

Hurray for VMWare! – I am using their products for years now – both private and on my job. It’s a blast to work with the Workstation and Fusion. Now they brought a major update to version 7 for VMWare Workstation and version 3 for VMWare Fusion. I upgraded my VMWare Fusion installation and finally the one feature that I missed the most on my Windows Vista and 7 virtual machines is available now: Aero Glas!

Oh… and it’s faster too

Source: http://www.vmware.com/de/products/fusion/index.html

If you – like us – need a picture of a shiny product box of a soon-to-be-released product for your presentation you may want to consider buying several tools to create such shots. But you can also just use a small tool and Windows Presentation Foundation.

There’s a great article on CodeProject where a almost everything is pre-set-up for our needs. And everything is written in C# – great stuff!

In action it looks like this:

Source: http://www.codeproject.com/KB/WPF/BoxShot.aspx?display=Print

It’s also suitable for anyone who wants to develop iPhone Applications.

“I started investigating how I might wire up — and then write native iPhone apps from — a scripting language. Lua was on my radar already. It’s compact, expressive, fast enough, and was designed to be embedded. Took only about 20 minutes to get the Lua interpreter running on the iPhone. The real work was to bridge Lua and all the Objective-C/CocoaTouch classes. The bridge had to work in two directions: it would need to be able to create CocoaTouch objects and also be able to respond to callbacks as part of the familiar delegate/protocol model.”

Source: Announcing iPhone WAX

There’s a great Visual Studio documentary on CH9. Highly recommended to anyone who wants to see what happened from the start till now.

“Welcome to the first installment of the Visual Studio Documentary.This is an hour long documentary that is split into two parts, roughly a half hour each. Welcome to part one, where we take you back to the days of MS-DOS and Alan Cooper who originally sold Visual Basic to Bill Gates back in 1988.  Next week we will feature Part Two but for those that would like to watch it sooner, here is Part Two. In addition, each week we will post a longer and more in-depth stand alone interview from the interviewees that were featured in the documentary.”

Source 1: Part I
Source 2: Part II

Almost three years ago I wrote about this nice little Regular Expression Tool which provides not only a RegEx-Builder but also a clean and nice interface to test and play.

It was a CodeProject sample project in that time and as it turns out it became a full blown version 3!

Obviously the user interface was revamped completely:

So you now not only get the Testing and playing but also a Regular Expression Library, a cool How-To, a more useable design mode and you can even output your final regular expressions to C#, VB.NET or managed C++!

Great stuff! Even better is the fact that it does not come at any costs. Despite the fact that there’s a registration you can just get your free license on their website.

Source 1: http://www.ultrapico.com/Expresso.htm
Source 2: want some espresso?

Great stuff this week: Notepad++ was released in  a new version 5.5. Nice new features all around:

Source 1: http://notepad-plus.sourceforge.net/uk/site.htm

Today we had a great meeting with SciEngines. These guys offer a great platform for everything that needs massive parallelism and IO bandwidth scalability. They even brought a small copacobana cluster to our headquater.

Source: http://www.sciengines.com

“ILMerge is a utility for merging multiple .NET assemblies into a single .NET assembly. It works on executables and DLLs alike and comes with several options for controlling the processing and format of the output. See the accompanying documentation for details.”

Have fun merging assemblies!

Source: MSDN

“So what is it? A memory mapped file allows you to reserve a region of address space and commit physical storage to a region (hmmm, sounds like virtual memory, isn’t it?) but the main difference is that the physical storage comes from a file that is already on the disk instead of the memory manager. I will say that it has two main purposes:

• It is ideal to access a data file on disk without performing file I/O operations and from buffering the file’s content. This works great when you deal with large data files.
• You can use memory mapped files to allow multiple processes running on the same machine to share data with each other.“

OMG! You can even specifiy views on a memory mapped file… from different processes… .NET 4 FTW!

Source: http://blogs.msdn.com/salvapatuel/archive/2009/06/08/working-with-memory-mapped-files-in-net-4.aspx

Well, if you don’t want to have them removed just form your Team Explorer in Visual Studio you want to go to your Team Foundation Server Remote Desktop and open a commandline.

Change to the folder %program files%\Microsoft Visual Studio 9.0\Common7\IDE and do this:

tfsdeleteproject /force /server: "
"

After the upgrade of all my machines to Windows 7 I now can write code for the new UI. Great stuff!

Microsoft today released the Windows 7 API Code for Microsoft.NET Framework on Code Gallery

Source: Code Gallery

It’s great to finally have the .NET sourcecode for debugging purposes – inconveniently it’s in a format you might have your difficulties just browsing along. A little tool is here to help!

After you installed, let’s say the WCF sourcecode and debug symbols you get a directory structure similar to this:

This source.zip.tmp file holds the whole sourcecode as one big package. It can’t be unpacked – even one would suggest that by just looking at that .zip ending in the name of the file.

Instead this is a plain-text file of a certain yet simple format. I wrote me a little tool to unpack this file into it’s original files and directories.

You can get the little tool, including sourcecode, here: UnpackMSSources.zip

To start the magic, you would like to go to the command line and start the tool with two parameters. Parameter 1 is the path and filename of the source.zip.tmp file. Parameter 2 is the part of the Path that needs to be cut-off. For the WCF Sources it’s “/DEVDIV/depot/DevDiv/releases/Orcas/SP/ndp/cdf/src/” for example.

The tool will then start to whirl through the file and extract all the files it founds into directories it’s creating along the way. After some seconds you would end with a directory tree like this:

Have fun!

Source 1: http://referencesource.microsoft.com/netframework.aspx
Source 2: http://www.schrankmonster.de/content/binary/UnpackMSSources.zip

We have this network share where each build from all the build-servers is dropped, including it’s test run results. It seems that we’re producing a huge number of almost empty filesystem test images which lead to astounding compression ratios:

If you got it, it’s easy. If you’re starting from scratch it ain’t as easy. We were in need of such a Username+Password Authentification so I started googling around.

I found several articles but had to mash it all together in a trial-and-error session. Now that I am enlightened I want to share my knowledge:

Step 1: Implement an UserNamePasswordValidator class and override the Validate method.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IdentityModel.Tokens;
using System.IdentityModel.Selectors;

namespace sones.Pandora.Database.Hosting
{
    public class UserNamePasswordAuthentification: UserNamePasswordValidator
    {

        public override void Validate(string userName, string password)
        {
            if ((userName != "Username") || (password != "Password"))
            {
                throw new SecurityTokenException("Validation Failed!");
            }
        }
    }
}

Step 2: Edit the App.config file to enable the previously implemented UsernamePasswordValidator.

    <bindings>
      <basicHttpBinding>
        <binding name="CustomAuthentication">
          <security mode="TransportCredentialOnly">
            <transport clientCredentialType="Basic" proxyCredentialType="Basic"/>
          security>
        binding>
      basicHttpBinding>
    bindings>
    <behaviors>
      <serviceBehaviors>
        <behavior name="SecurityBehavior">
          <serviceCredentials>
            <userNameAuthentication
            userNamePasswordValidationMode="Custom"
            customUserNamePasswordValidatorType="sones.Pandora.Database.Hosting.UserNamePasswordAuthentification, PandoraDB_WebServiceHost_UsernamePasswordAuth"/>
          serviceCredentials>
        behavior>
      serviceBehaviors>
    behaviors>
    <services>
      <service behaviorConfiguration="SecurityBehavior" name="sones.Pandora.Database.Hosting.PandoraDatabaseHost">
        <endpoint address="" binding="basicHttpBinding" bindingConfiguration="CustomAuthentication"
          name="ep1" contract="sones.Pandora.Database.Hosting.IPandoraDatabaseHost" />
      service>
    services>

In this example the ServiceHost will use no server SSL certificate and therefor allow normal http access instead of just using https ssl. You can configure that behavior with the <security mode=”TransportCredentialOnly”> line. Just change there and define an apropriate certificate and you’re good to go with https / ssl.

I’ve seen the launch of PopFly two years ago. And now I am going to see the landing

“Unfortunately, on August 24, 2009 the Popfly service will be discontinued and all sites, references, and resources will be taken down. At that time, your access to your Popfly account, including any games and mashups that you have created, will be discontinued.”

Now somebody please tell the world that the code of popfly will be released in some way so that other people can learn and work with it. That would be great. Oh if we just wouldn’t have that many lawyers on this planet.

Source 1: http://popflyteam.spaces.live.com/blog/cns!51018025071FD37F!336.entry
Source 2: http://www.popfly.com/

Ich hatte Ende letzten Jahres ja schon einmal ein Stellenangebot hier online gestellt. Damals mit dem Ergebnis einige sehr interessante Bewerber und letzlich auch hochmotivierte und qualifizierte Mitarbeiter gefunden zu haben.

Da wir nun wieder auf der Suche nach Verstärkung sind nutze ich wieder dieses Medium:

Die sones GmbH ist ein junges IT-Unternehmen mit Standort in Erfurt. Wir forschen in den Bereichen neuartiger Datenbank- und Speichertechnologien und entwickeln auf dieser Basis neue und innovative Produkte und Lösungen.

Am Standort Erfurt suchen wir ab sofort eine(n)

Software-Entwickler JAVA / .NET (m/w)

Sie wollen in einem jungen Team innovative Software entwickeln die im Datenbank-Segment ganz neue Wege aufzeigt? Als Software-Entwickler bei der sones GmbH haben Sie hierzu die Gelegenheit!

In einem hoch motiviertem Entwicklerteam arbeiten Sie am Kern unseres Datenbanksystems mit. Sie entwickeln Features und verbessern die Qualität der Codebasis im Hinblick auf Stabilität, Performance und Skalierbarkeit. Dabei kommen modernste Entwicklungswerkzeuge zum Einsatz.

Wenn Sie unsere hohen Ansprüche an fachliches Wissen, Eigeninitiative und Kommunikation als Herausforderung sehen – dann sind Sie bei uns herzlich willkommen!

Ihre Aufgaben:

• Projektplanung und Projektsteuerung in Koordination mit anderen Entwicklungsbereichen
• Analyse, Design, Implementierung neuer Produktfeatures
• Verbesserung der Qualität existierenden Codes im Hinblick auf Stabilität, Performance und Skalierbarkeit
• Softwaretests und Dokumentationen
• Evaluierung neuer Technologien und Prototyping

Voraussetzungen:

• Studium im Bereich der Informatik oder vergleichbare Ausbildung mit überzeugenden Referenzen (Projekte, Beschäftigungen)
• Mehrjährige Erfahrung in der Objektorientierten Softwareentwicklung
• Von Vorteil:
  • Programmierkenntnisse JAVA und .NET
  • Erfahrungen mit Testdriven Development
  • Gute Englischkenntnisse
  • Erfahrungen mit Datenbankarchitekturen und Netzwerkprogrammierung

Ihre Soft Skills:

• Kommunikationsstärke und Bereitschaft zum dynamischen Wissens- und Informationsaustausch
• Zuverlässigkeit und eigenständige kreative Denk- und Arbeitsweise
• Ziel- bzw. Lösungsorientiertes Vorgehen

Wir bieten:

• Hoch motiviertes und qualifiziertes Team
• Ausgesprochen interessante und innovative Arbeitsgebiete
• Viel Platz für Eigeninitiative und Kreativität
• Die ständige Möglichkeit sich weiterzubilden und weiterzuentwickeln
• Herausforderndes Umfeld eines High-Tech Start-Ups

Sie sind interessiert? Dann freuen wir uns über ihre aussagekräftige Bewerbung mit Angabe ihrer Gehaltsvorstellung an jobs@sones.de

Der Vollständigkeit halber das Stellenangebot nochmal als PDF:

Stellenangebot sones GmbH

If you’re writing WCF Web Services you maybe came to the point when you needed Administrator rights to start the Web ServiceHost. As a matter of fact the only thing you need is the right to use a that URL space.

So for  a WCF Web Service running on http://localhost:80/TestService/Ep2 you would use the netsh command line tool to set the correct rights.

Step 1: Start an Administrator-Commandline

Step 2: run “netsh http add urlacl url=http://+80/TestService/Ep2 user=SONES\bietiekay”

(SONES\bietiekay = the Domain+User to grant the right)

I was in desperate need for an DDate equivalent running on Windows. DDate is an unix implementaion of date accoridng to the erisian calendar described in the principia discordia.

I only found some C Implementations. And since it’s fun to do I ported the original Discordian Date C code to C#.

You can download the C# sourcecode, licensed under CC-BY-NC here.

I also created a web page which displays the current discordian date and offers you to convert any gregorian date into discordian date representation.

This page can be accesses here. You can call another page with parameters and you only will get the ddate output back:

for example: http://ddate.schrankmonster.de/DiscordianDate.aspx?year=2009&month=6&day=9

Source 1: http://ddate.schrankmonster.de/
Source 2: http://dropbox.schrankmonster.de/dropped/SharpDDateLib.zip

There’s an update of the beloved TFS Build Status Screen tool. And the most frequently asked feature is now built-in: Scaleability.

You can scale the status screen now to fut even on the smallest screens…hurray!

Source: http://raquila.com/software/team-foundation-server-build-notification-screen/

Oh dear. Another hyped protocol/platform from Google… oh wait. It’s not from Google. It’ all started in Xerox PARC…

There are several papers that describe what Google now claims to have developed…

left: Xerox PARC Paper; right: Google Wave

Conclusion: Go and read old Papers. As it turns out almost all newly hyped things have been described in papers from years ago.

Source 1: http://www.waveprotocol.org/whitepapers/operational-transform
Source 2: http://doi.acm.org/10.1145/215585.215706

We’re currently running several build processes. So each time someone checks new code in one of the build machines gets the whole package and builds it, runs tests on it and stores the result of this whole process on the Team Foundation Server. Great stuff so far.

Until you start to do things like automated WCF Testing. We’re using the selfhosting capabilities of the WCF to start a ServiceHost and then run tests against it. This works great locally. It does not on the build machines. Even if you promote the Build-Service User to Administrator you won’t get the love.

The error you might get would look something like this:

The exception contains an URL which tells you to add the Service URL to the machines URL Access Control List. On Windows XP and 2003 you have to install the Windows Support Tools and use the httpcfg command. On Windows Vista and 2008 you should use the already installed netsh commandline tool.

Since we need to get this to work on all current and future build servers I decided to add the netsh call to the build script, which looks like this:

” border=”0″ alt=”” src=”http://www.schrankmonster.de/content/binary/WindowsLiveWriter/get.AddressAccessDeniedExceptioninautoma_9859/Capture2_thumb.png” width=”400″ height=”109″ />

Add this Target before any tests in the .proj file and you’re set.

Source 1: http://go.microsoft.com/fwlink/?LinkId=70353

Aufgrund neuester Entwicklungen im Speichermedien-Segment wird ab dem nächsten Release des sones Speichersystems auch das angesagteste Speichermedium der Stunde unterstützt: die Speichergurke.

Durch die sensationelle Speicherdichte und unerreichte Zuverlässigkeit ist die Speichergurke das perfekte Speichermedium für den Datenhunger von gestern, heute und morgen.

Source 1: http://www.sones.de
Source 2: http://www.speichergurke.de

Turns out that we’re actually doing business there – great news that is!

Source: http://www.next-conference.com/next09/