Time

Recently I talked with the CTO of a major systems provider. At one point I used the term "generic" together with "software development" and wanted to carry on saying that Open Source also does this and so on. But I never got there. He stopped me and said that I should please never use the term "generic" again. As he carried on he could barely contain his anger, not at me but at the rest of the world of software development, sort of. He had heard the promise of generic software so often and it never worked out, that he just got tired of it. He explained that especially in the Web context he explicitly goes down another route these days. He wants a one-off development that will do just exactly what he needs for two or three years. Then he will throw it away and get something new. It is always a one-off, never more. He does not want any documentation, no upgrade plan, no support, he wants a running system. Full stop. That was news to me but it got me thinking.

The Lifetime of Geodata

Now, especially in the geospatial domain things are a bit different. Geodata has a very very long life time. Not all but some geodata almost seems to live forever. Be it satellite images that need to be archived - and then read a decade later, or cadastral data that evolves as people buy and sell land, cut up land lots and join them up. It is almost a living system. In many cases the lifetime of the data we work with survives our employment. The data was there when we started and it will be there when we leave. Some data will outlast our own lifetime. Ugh. Face it.

The Lifetime of Software

So how do these two things go together? The lifetime of (certain) software becoming shorter and shorter and that of data extends practically endlessly? Right now it seems to not go together at all. One of the primary reasons is that software developers tend to perceive data as a nuisance. Something that gets in the way of great software design. It has to be taken into consideration but it is not fun. It slows innovation and takes up resources. Go away.

Software developers are bad Data designer

What is the source of this problem? It should be blatantly obvious by now. Typically the people implementing software are software developers. Did you ever meet a data developer? No? Me neither. And if you did, are you sure it really was a data developer? I bet it probably only was a failed software developer. Or someone infected with an architecture design, be it Client/Server, SOA, Web 2.0, RESTful, API or whatever. What do they have in common? They think in terms of the software. The only recent exception may be resource orientation or the ROA (Resource Oriented Architecture). But do you read anything much about this anywhere?

Metadata Anybody?

Boohoo! Boring! Dusty! Go away! This is because we lack a data culture. There is a disconnect between the needs of a data and the people who could actually provide it. Software developers think in terms of their software. Architects think in terms of scalability, security, maintenance - of the software. Application designer think in terms of user experience. And data - is just there. A never ending nuisance full of (real world) errors and never quite finished. Ever evolving. Painful for someone who wants to design a generic software.

This is also the reason why we have no metadata. Anybody really in love with their data would maintain it's lineage. It would be beautiful metadata. Not pointy brackets we have to fill with predefined content like INSPIRE wants us to do. We would document where the data came from, what happened to it, who worked on it, what is the accuracy and so on. But nobody cares about this. Because you need a pretty technical background to understand how to do it. And the people with technical background usually come from a software perspective. It is a deadlock situation. Praise Open Data if only to bring out this problem into bright sunlight for everybody to see.

How to Proceed?

We developed a few ideas during my recent stint at the Ordnance Survey. Thanks agin to everybody who I worked with there for a very educative and insightful time! What we did is take a piece of geospatial data and track it down throughout the organization. Ignore the software. Just follow the data. And then look at how this path can be improved. From the perspective of the data. Amazingly it can be improved by simply removing half the software. And by maintaining a record of what has been done to the data as it travels through the organization. And by talking to the people who did the little dirty tricks to the data to make it work with "their" software. Did you hear that? The things that were done to the data to make it work with the software. This is simply wrong. It should be the other way round. You should never, ever change your data according to the specs of a software. Data must come first, because:

Your software will go away. Your data is going to stay.

Have fun,
Arnulf

Social networks are great. If you know how to use them. I apparently don't. I forgot to update my notification email and thus received no more notifications including those that notified me of not being reachable anymore. This makes all the great social networks pretty useless if seen just on their own. What makes them work is that there are more than just one. Redundancy allows me to find out about a problem in one of my self maintained contacts. ((This is why Google, Facebook, PayPal, eBay and so on can make a living. Because we work for them. We maintain our own contact information for them. In turn we get. Well, something, else we wouldn't bother, would we?))

By the way: When will those places loose their Unique Selling Proposition? When there is only one o them left. Because monopolies kill themselves. After prospering for some time. They eventually suffocate. Always. Just like large corporations. Even the one largest corporations in history ever is right now in a deep crisis, church. But that is another story.



Social networks will simply stop to be a business model out of themselves when people understand how to stay in touch with each other without falling back to paper on a notebook.

What do we need to do that? We need one unique ID. That's it, just one. How do I know.? Because I was blessed (and doomed) to have been awarded a unique ID when my parents chose my name. "Arnulf Albrecht Benno Christl" is - as far as I know - a unique ID on the Web. Search for it and you end up on my digital doorstep. Then you have the choice how to contact me. There are an some 50 ways to get my attention. There is my private web site and the business site and some dozen project web sites which all contain one or another way of contacting me, either through a mailing list (where I have subscribed) or an IRC chat (which I monitor) or a blog comment field (which notifies me via email) or Facebook or LinkedIn or Xing (the last one was the one I forgot to inform about my new address) and so on. And if I ever do loose my password I tell themachine to send me a new one. Or a link to a new one.

Now, what would you do if you had no ID on the web yet? Imagine that never before have you ever been in the Web. What is your ID on the Web anyway? Are you an individual? Meaning indivisible. If you have one, what is it? Now comes the gist of the story. It is your email. Strangely enough, it is the only medium that reliably stitches all together. SMTP. Simple as that. Without email my Web would immediately start to rot. This is such a banal insight that it is not even worth bothering you with it.

But then again, what does this mean to how the Web an we interact?

Ah, and one last thought. If we would all use PGP keys then we could even communicate in private. Because all this privacy crap about corporations spying on us is rubbish. As long as we use SMTP to communicate with each other - without encrypting it – we are open like open books. But that is just as banal an insight as the one before.

Have a great weekend!