Space

Recently I had an interesting thought about the Web. To the knowledgeable of us this is probably obvious but anyway. Just ignore me if it bores you.

The Internet is a Hierarchy

The technical foundation of the Web is the Internet. It consists of a domain naming system and is typically three levels deep. The Internet top-level domain (which should be called the root-level?) is a two or three letter string as in com, org, de or us. For some reason they are not directly addressable (as empty nodes?). Below (or rather: growing from there) are the domain names as in osgeo.org or w3.org and they are addressable. Below (or rather on top) we oftentimes add a pretty superfluous "www" as in http://www.gov.vu/. Sub domain piling as in http://inspire.jrc.ec.europa.eu/ typically visualizes over structured organizational mammoths. We can add directories to the other end of the string. Old sites make a habit of organizing their content in useless and empty directories like in http://www.osgeo.org/content/sponsorship/sponsors.html which is the identical resource as in http://www.osgeo.org/sponsors. But essentially there is no difference in how we structure them (think Wikipedia where everything is on exactly one(!) level, except for categories and some special pages).

One can display this as a site map which looks almost like an uprooted tree growing from the top downwards. There are really big sitemaps but they are still flat, two levels, maybe three.

The Internet is Flat

The Internet is a really flat hierarchy. Looking at the Internet horizontally makes this obvious. There are a billion web sites and most of them are typically three levels "deep" (or "high") and almost never more than seven levels. It reminds me of our beautiful planet earth. The highest elevation above sea level is a mere vertical 10km. Compared to the 40000 km circumference this is not much at all. And look at our breath after climbing a flight of stairs (something like 0.001 km).

XML

Back to technology: XML is a commonly accepted format to represent trees and hierarchies. Each web site structure can be represented as an XML tree. If you put them all together and call the root "Internet" then you have one single tree of the whole Web. But this type of representation does not help us at all because it is too "wide" and "flat" to be useful. Additionally there are no directions and there is no way to find ourself around. Which is why we still depend on search engines so much.

The Web is a Graph

Lets switch perspectives and go to the Web (keeping in mind that it runs on the clear hierarchical structure of the Internet). On the Web we can typically follow links to other sites. We can jump (or surf) from any level and branch and leaf of one tree (web site) to any branch or leaf on any other tree just like Tarzan of the apes - provided they are linked.

This is pretty crazy. We can hop through billions of web sites without ever having to climb down one tree and up the other. We can hop from one top branch to the next. Where is the intricate bit? What makes this possible? We can only jump from any tree to any other tree like Tarzan of the apes when we have links. This is actually the most important aspect - Hyperlinks.

Links are a bit more complicated and differ to trees because they are always directed. We can go back through our browser history, but not necessarily anybody else (except we carelessly or consciously allow Google, Facebook, Bing, Yahoo, eBay and the rest of the gang to eavesdrop on us and record whatever we do). With a tree this is different because going up and down does not make much of a difference (except if you are a hapless cat). And you can only come down a tree if you went up before. But you can follow this link from my little home here to the home of the oracle turing machine but you will surely never ever find a way to jump back here from there. Which is inherently different from hurrying up and down trees where you will always have to go down one tree before you can hurry up the next. Actually there is a way to climb back but it is tied to the state of the representation (in the referrer URL) and the path of the surfer, but it is also private - but to the destination web site. Yes, intricate.

Now what? We have this great XML technology to represent the hierarchy of the domain naming system but it fails hopelessly if we want to use it to hop to another branch or leaf directly. Because the Web is a directed graph.

The Web is Truly Multidimensional

This is where my bulb started to shine and then blew. Because it goes beyond what my little mind can handle. The Web is truly multidimensional. And to be honest, I already have problems with 3 dimensions.



Two and a half dimensions are all that my mind can confidently handle. Now you might say that we live in a three dimensional world - but do we really? Is it not just surfaces? And then we have time which could be seen as a fourth or fifth dimension - but we are pretty much stuck in the present and move through time involuntarily. Only our mind can travel time to some degree but that is a different story. Moving to the fifth or sixth dimension we are definitely lost. The Web has billions of dimensions. Literally.

RDF

So what can we do about this? For a change we could explore the graph that represents the Web. The technology that invariably comes up is RDF and with it the concepts around semantics or the semantic web. Which is something that I now understand is not to be implemented but already there - only that we have no common way to really comprehend or represent it visually. Fortunately the graph and RDF and triple stores can be represented using XML, which brings us back to our holy hierarchy and things we do understand. The path out of our informational doom is probably tied closely to exploring the graph. But there is no way to visualize the graph as a whole. We can get a two-dimensional representation as shown in an example right on the face of the Linked Data web site. To better understand what the graph of the Web can give us we need to make it intelligible by twisting it and then cutting it into planes creating two-dimensional cross sections. Once there we can follow with our two dimensional minds and explore nodes by cutting new cross section planes.

Maps are structured Geographical Data

I can see a certain analogy here. What always intrigued me is that maps seem to bend the Web back down to physical locations on earth. And in my perception it is high time to redesign navigating maps - or rather geographical data - to resemble how we browse the Web. Catalogs and structured metadata fall short of addressing this need, thought they might be a handy asset. We need an open perception of data and meta and how they intermingle and link... Ah - there was that magic word again. Link it, babe.

Same for directory trees. They are good for structured layers in maps but not suitable at all for arbitrary geographical content. We should move away from folders towards dynamic graphs. Pick a node and display it in a growing map context. Simple as that. The idea is far from brilliant and many others will have brought this up before but so far I did not find a convincing implementation yet. If you have, please let me know.

Have fun,
Arnulf.

In case you have not heard about it yet: There will be an Open Source Mash-Up workshop at the INSPIRE conference in Kraków, Poland under the motto "biodiversity". All and everybody is invited to take part, be it at the conference or remotely.

• What is going to happen?
• Whatever you can make happen...

How did it come into being? Paul Smits from the JRC approached Andrea Giacomelli and me at the last OGC TC meeting in Frascati, Italy and asked whether we would be interested in helping to organize an Open Source Mash-Up at the upcoming INSPIRE conference in Krakow. We both confirmed that this is an excellent idea and that we would be delighted to help organize it. Andrea suggested that a motto would help to get people interested and came up with "Biodiversity". He started to put together the description of the event and launched a call for participation. Old bad me did not do anything much except talk to people about it wherever I met them because I have business (but that is another story).

Who is organizing this? Currently "we" are the JRC and OSGeo and we try to address a broad group of stakeholders ranging from the lone techie developer through environmental experts with a spatial problem on their hands right up to the business executive of the European NMCAs (National Mapping and Cadastre Agencies), EEA (European Environmental Agency) and whoever else thinks that they can contribute or take away something. It will be interesting to understand what the the motivations of the participants are but that angle of a stakeholder analysis will have to wait until after the event. Lets have a look at the initiating parties.

What is the JRC? It is the "Joint Research Center" and no, they do not research the effects of THC on scientist's brains but they "enable" INSPIRE. They have been tasked by the commission to provide an infrastructure and organizational framework enabling experts, users and managers to design the European (environmental) SDI. Mind me: INSPIRE is not the JRC! INSPIRE is aprocess. Regardless of what this means in detail it has become obvious that the few INSPIRE folks in JRC are hopelessly over tasked. Organizing the design of Europe's spatial data is a Titan's job. No matter how good and industrious the INSPIRE crew is, they are humans, not Titans. Europe's spatial data as a whole is way to large, heterogeneous and non-ingeniously organized to get anywhere fast. It has grown literally over hundreds of years and with IT becoming commonplace a legacy of technological silos emerged. These we have to unite.

I guess that most of those who end up reading this Blog are pretty aware of what OSGeo is. Beyond the task of providing a coding infrastructure, QAing project governance and organizing the FOSS4G conferences OSGeo also supports free and open access to geospatial data. This is where OSGeo should and can hook into INSPIRE. OSGeo's spatial IT know-how can tie into INSPIRE in many different ways. Jo Walsh registered OSGeo as an SDIC (Spatial Data Interest Community) in the INSPIRE process, so "we" have a voice there. But afaik no one is talking (except for the odd complaint on Twitter or standards ML that INSPIRE will never work or is all wrong). This may be one of the core issues with INSPIRE. It is broadly recognized as a legal framework, but it is not seen as a process that is open for participation. It has started as an idea at the end of the last millennium and has grown in many different ways. One is the regular standards based, organized, legally binding sometimes sore process of defining specifications and guidelines. Some of this process and their results have been criticized heavily by "us" (that is the agile, sleek, geek "us"). Some LMOs (the Legally Mandated Organizations in INSPIRE, for example the NMCAs) have bent and obstructed INSPIRE until they succeeded in removing some of the explicit openness that was core to INSPIRE since its inception. That is when the GeoRM layer invaded the architecture diagrams, prominently "protecting" all within. We might get a taste of this during the Mash-Up workshop when we try to access some of the data and map services that are made available through the INSPIRE framework. Even if the only outcome of our Open Source Mash-Up event is that the RM crap prevented anything from getting done, it would still be a good result because it gives us a chance to show whats wrong and not only talk about it.

Remember: INSPIRE is neither a physical thing nor a person nor a single organization. It is but an idea and ideas are worth nothing until they are realized. If policy, GeoRM, crappy service quality or technological excesses prevent INSPIRE from taking off then we will have to remove or steer around GeoRM, improve the service quality and propose alternative technology. I say "we" because I believe that it will involve all parties including NMCAs, JRC, OSGeo, OGC (just to name a few) but also the "broad public".

Yes, INSPIRE also foresees interaction with the "broad public". What is the broad public? Some understand it as a Mass Market as it is addressed by the GoogleBingYahoos of this world. Others believe in more collaborative and inclusive governing by the people themselves and call the broad public "European citizens". Think e-Government not as online forms but as collaborating on governing. There are a few hundred million people around and I guess there should be one or two who are also spatially aware and potentially even technologically savvy enough to get something done. But how to identify and then interact with them?

In the ideal case developers and users will meet in a workshop room and collaborate on Mash-Ups. What could that be? I am pretty sure that a few OpenLayers based applications will pop up and maybe a prototype for the biodiversity folks. We might learn that OpenLayers and GDAL/OGR will need to learn GML 3.2.1 just as deegree has. Maybe we just get a better understanding of what INSPIRE is and what it is not.

So lets actually mess up INSPIRE a bit. Interested? Then go register at the INSPIRE Forum and find out what is already going on. Support in action.

If you have questions, post them here, the OSGeo Discuss mailing list, the INSPIRE forum or hassle me directly. Unfortunately I wont be at the conference myself (because they rejected my submission with the title "How INSPIRE failed on metadata"... . But Paul will be there and Andrea and several other well-known binaries – I mean: folks. And I promise that it will be a productive event - no matter what comes out of it. It will surely be more than what we have achieved so far.

Have fun.

Udpate

Read about the first results of the workshop / mash-up in the INSPIRE forum.

Disclaimer: This is no more than a brain dump.


There are some obvious issues with Open Source in the EU and INSPIRE that have been troubling me for some while. For some reason no one inside the commission who I have talked to seems to get this which makes me wonder whether it is me who got it all wrong. So bear with me for a moment and feel free to share your thoughts.


As I understand the INSPIRE process, it should be influenced by three distinct parties: The public administration, private companies with a business interest and the public at large. From all that I can tell this latter group includes grass-roots Open Source projects and those sustained by small businesses. I am not talking about the big funded (and sometimes somewhat bloated) projects, those are another story.


The active participation in the INSPIRE process involves quite a bit of monetary and temporal investment. In most cases this investment will only be done only by two of the three parties. One is the public administration, they are obliged to take part as best as they can and it should be in their best interest. But – we are talking hierarchy and top down processes in legal frameworks – not bottom up processes that would actually work. The second active party are companies with a sizable enough budget to fund up-front investment. They only move if they can expect a return on their investment. This is perfectly in order in our current economic system. But it also means that real input to the INSPIRE process is restricted to two of the three parties. Those who have a motivation to make a profit and the public administration.


Larger profit driven companies are known to miss the point by not listening to their users or going astray with queer technology and building vendor-lock-in. Not much innovation to expect here. The public administration, government institutions are routinely understaffed and – no offense taken – typically undereducated. This is the result of years of privatizing institutions, a process we all support when we vote for governments (but that is another story).


The third group - the public at large - is only theoretically considered but cannot get involved in any noticeable way due to restrictions in time and money. In my typically skewed perspective this is also where innovative Open Source could but does not come into play.


This flaw is a lot more blatantly visible in all EU funded projects - which again makes me wonder why it is not more prominently recognized or discussed. Let me explain, it is a really simple logic: Work packages in EU funded projects typically require private investment of 50%. The return on this investment is supposed to come from selling "something". In the software domain common understanding still seems to generate revenue by charging usage fees for proprietary licenses. This obviously does not work for Open Source because once the work is done, there is no more incentive to pay for it.


Funny enough lately Open Source has become a buzz word in EU grant applications. But no more than that. The result of EU funded projects consequently are so many software hacks in a wild state of incompleteness and lack of real world usage. Again – no offense meant – it is the system that makes us do it this way. The main reason why most "Open Source" projects fail is that they spell "Open Source" in quotes. They fail to follow the Open-Source-way-of-doing-things right from the start (read up on http://producingoss.org to get the gist). Instead in many cases it feels more like a random license has been stuck on top of the project at the last moment before submitting the final report. This is not just a waste of resources but it also damages the reputation of those who implement quality software under an Open Source license.


As already stated at the outset this is nothing but a brain dump and not meant to put anyone off. My guess is that most of these issues simply result from a deep misunderstanding and thorough ignorance as to what makes Open Source projects function successfully. But who on earth will pay someone like me to educate folks before submitting a grant application or talk to the commission in a way "it" understands? Some already do, and there is OSOR for a start, but it is still a long ways to go. Meanwhile INSPIRE goes – well, where exactly? I'll go ask some Borg when I am back in orbit, they should know, they know how to communicate.


Have fun.

When Christoph Baudson and me prepared the presentation Typification of Web GIS Clients (and English language version will follow) we wondered how many developers would be at FOSSGIS. And we came up with not even an handful and we know most if not all of them by name. While this is great we would really like to make FOSSGIS more attractive to potential new developers. This is why the opening lightning talks were carefully chosen to get more techies interested. They ranged from "How to contribute to Open Source software projects" by Andreas Hocevar, through "Browser-based Gaming with OGC services" by Thorsten Bockmühl, up (or down) to "NoSql" by Volker Mische.


The keynote was awarded to Marc Jansen who talked about "ECMAScript, HTML5 and the Cloud" and allowed us a glimpse into the future Web which really looks quite bright.


This year all of this had a bit of an aspect of casting pearls before swines but maybe word of mouth will help to get a more tech focus next year.


Have fun.

Let me introduce my new brand Metaspatial. The name Metaspatial combines "meta" which is derived from Greek: μετά and means "after", "beyond", "with", "adjacent", or "self". It is a prefix used to indicate a concept which is an abstraction from another concept, used to complete or add to the latter. Now that I come to think of it, the name of my last project might be Postspatial...


Whatever, it feels great to head on. A big thanks goes to all my colleagues at the WhereGroup who have always supported me generously in my quest for spatial enlightenment and endured all the standards and free software model escapades that got us where we are now. It was a great time and there is a saying that that is the time when one has to leave.


I will spend my newly found free time to... but wait. Didn't I say that I have no time? And how can one "spend time" anyway. Maybe we are more like granted time. Reformulating: My newly granted time will go into the physics of space time and mind.


So what has changed for me? Nothing much. And for you? Nothing much either. I am still open for software and architecture consulting, prototyping, agile development and spatial (meta) data - with one little catch: You need to get me interested.


Have fun.

Update

As of June 2010 the logo of Metaspatial has changed (see below for the old one). I finally found time to talk to my designer and for once I listened to her. So here we go with a new logo. How do you like it? If you are interessted in the background, feel free to read the About pages on Metaspatial.

If I would really want to know then I should probably make a new blog entry and not eidt an old one that no one will ever look at again. Now I wonder: Should I remove or change the old blog or leave as is. Seems that everybody handles that differently. Some say a blog has to stay unedited, others edit freely, even weeks and months later. Dunno. What do you think? (Just ropened commenting for a while.)