OSGeo - Powering SDIs globally - GSDI 2009 Paper 272

From Seven
Jump to: navigation, search

Paper submitted to GSDI 2009, download ODT or PDF version.

Contents

OSGeo - Powering SDIs globally

This paper introduces to the activities of the Open Source Geospatial community and its role in supporting spatial data infrastructures around the world. Open Source software is used pervasively throughout all domains of IT ranging from operating systems to web applications. The Open Source spatial domain has grown over the past years producing many good software packages, some of which have developed into reliable core components that are used to build a strong and comprehensive spatial data infrastructure. The natural use of standards as a means to connect and interoparate with other packages has helped to build a strong community of professionals from all domains.

Introduction to OSGeo

The Open Source Geospatial Foundation (OSGeo) is a not-for-profit organization founded in 2006 whose mission is to support and promote the collaborative development of open geospatial technologies and data. The foundation provides a collaboration environment, organizational and legal support to the broader open source geospatial community. It serves as an independent legal entity to which community members can contribute code, funding and other resources, secure in the knowledge that their contributions will be maintained for public benefit. To ensure proper governance

OSGeo also serves as an outreach and advocacy organization for the Open Source geospatial community. It provides a common forum for users, businesses and developers of Open Source geospatial software. OSGeo operates a comprehensive development infrastructure for its projects ensuring highest quality software and improving cross-project collaboration.

The foundation's projects are all freely available and useable under a license certified by the Open Source Initiative (OSI). Open Source software projects can apply to go through the OSGeo Incubation process and become an official OSGeo Project.

The Incubation process

The purpose of the OSGeo incubation process is to ensure that projects follow the basic official OSGeo guidelines on Open Source software development[1].

This means that they:

  • have a successfully operating open and collaborative development community
  • have clear IP oversight of the code base of the project
  • adopt the OSGeo principles and operating principles
  • are mentored through the incubation process

Open Source software projects originate from very different backgrounds and cover all types of programming environments (Java, C++, Python, JavaScript, etc.). Some have been Open Source right from the start and evolved over many years like the GRASS projects that was started more than 20 years ago. Other projects have been published under an Open Source license right from the start but were initially managed by a single commercial entity like Mapguide Open Source by Autodesk where the development and governance structures were only subsequently opened to the community. Yet others like Mapbender have started off as proprietary software and were later released under an Open Source license when the corresponding vendor started to understand the potential of the Open Source development methodology. Some like deegree started in a cooperation between academia (University of Bonn, Gemany) and a commercial entity (lat/lon).

All these different types of software and styles of governance have to adhere to the same basic Open Source principals as set up by OSGeo. Instead of inventing all of these principals from scratch the bascis have been gleaned and from successful Open Source initiatives like the Apache Foundation. To be able to account for the specialities of the existing diversity of projects they have been adapted and honed to suit the need of asa diverse community as that of the geospatial realm. The resulting guidelines specify that during incubation and after:

  • Projects should manage themselves, striving for consensus and encouraging participation from all contributors - from beginning users to advanced developers
  • Contributors are the scarce resource and successful projects court and encourage them
  • Projects are encouraged to adopt open standards and collaborate with other OSGeo projects
  • Projects are responsible for reviewing and controlling their code bases to insure the integrity of the open source baselines.

A designated mentor from the OSGeo Incubation Committee accompanies the project through the processes and helps to solve any problems. Completing the Incubation Process can take several months of work depending on the size of the project, legal background of the code base. When all issues that were identified as critical to a healthy project have been addressed the mentor proposes the project for graduation. Then the Incubation Committee consisting of currently 18 members from different backgrounds goes through the documentation of the project's incubation and brings up any open ends to ensure highest quality. At the next Board Meeting the directors of OSGeo have a formal vote to graduate the project after which it becomes an official OSGeo project.

All the details of the OSGeo Incubation Process are available at the OSGeo Wiki website at: http://wiki.osgeo.org/wiki/Incubation_Committee

The OSGeo SDI Stack

Currently OSGeo references 16 projects (March 2009) including desktop applications, web mapping software, geospatial libraries and meta data catalogs. Some projects cover more than just one domain (deegree, MapGuide Open Source), others are highly focused on one single target (MetaCRS, GDAL,OGR). Implementations come in all major programming languages including Java, C++, PHP, JavaScript and others and most are cross platform compatible running on Linux, UNIX, Mac OSX and Microsoft Windows platforms.

This paper will focus on the OSGeo SDI software stack as listed on the web site under Web Mapping (http://www.osgeo.org/). This stack is complemented by spatial Open Source components like PostGIS (http://www.postgis.org/) that are not (yet) part of the OSGeo stack and non-spatial software like operating systems, web servers (http://www.apache.org/) and databases (http://www.postgres.org/).

deegree

deegree supplies the building blocks of a Spatial Data Infrastructure, while implementing the standards of the Open Geospatial Consortium (OGC) and ISO/TC 211. The Java-based deegree Framework is the most extensive implementation of OGC/ISO standards in the field of Free Software.

Features

deegree comprises five product groups of which three are directly related to SDI:

  • deegree Web Services: the base components of any SDI; including WMS, WFS, WCS, CS-W, Gazetteer, WPS, WCTS, WTS/WPVS, security services like WAS and WSS, WMPS (deegree-specific) and owsWatch (service monitoring).
  • deegree iGeoPortal: the portal framework of the deegree project. iGeoPortal has a modular structure and comprises so far the modules map, gazetteer, catalogue, security, data and 3D.
  • deegree iGeoSecurity: to keep your SDI secure

Mapbender

Mapbender is an SDI back office to manage, monitor, catalog and visualize services. It contains a framework to deploy web based SDI applications and gives access to the OSGeo SDI stack. End-user interfaces can be highly focused with only the minimally required functionality or full-fledged applications with security proxying, WFS-T digitizing with auto snapping and more.

Comprehensive web based management interfaces for the administration of the SDI allow to maintain and categorize map and feature services and grant access to individuals, groups and other services. Adherence to standardized services, such as WMS and transactional WFS, allow to take advantage of interoperable services from various server platforms.

Features

  • Software and management services for mapping of OGC web services architectures
  • Implements the latest web technologies using PHP, JavaScript and XML
  • Provides a data model and interfaces for displaying, navigating and querying OGC compliant map services
  • Authentication and authorization services
  • Security proxy functionality
  • Management interfaces for user, group and service administration


MapGuide Open Source

MapGuide Open Source is a web-based platform that enables users to quickly develop and deploy web mapping applications and geospatial web services. MapGuide features an interactive viewer that includes support for feature selection, property inspection, map tips, and operations such as buffer, select within, and measure. It includes an XML database for managing content, and supports most popular geospatial file formats, databases, and standards. It can be deployed on Linux and Windows, supports Apache and IIS web servers, and offers extensive PHP, .NET, Java, and JavaScript APIs for application development. MapGuide Open Source is licensed under the LGPL.

Features

  • Windows and Linux, Apache and IIS, and multiple browser support
  • Interactive map viewing
  • Quality cartographic output
  • Built-in storage of XML resource documents
  • Uniform access of raster/vector data using Feature Data Objects (FDO) API
  • Flexible application development: PHP, .NET, Java
  • Extensive server-side APIs
  • Fast, scalable, secure server platform

MapServer

MapServer is an Open Source development environment for building spatially-enabled web mapping applications and services. It is fast, flexible, reliable and can be integrated into just about any GIS environment. Originally developed at the University of Minnesota, MapServer is now an OSGeo project and is maintained by developers around the world.

MapServer runs on all major operating systems and will work with almost any web server. MapServer features MapScript, a powerful scripting environment that supports many popular languages including PHP, Python, Perl, C# and Java. Using MapScript makes it fast and easy to build complex geospatial web applications.

Features

  • Supports industry standard data formats and spatial databases
  • On-the-fly feature classification
  • Sophisticated rule-based labeling
  • On-the-fly projection for both raster and vector data
  • Provides a wide variety of spatial and attribute-based queries
  • Supports popular Open Geospatial Consortium (OGC) standards including WMS, WFS and WCS
  • Leverages best-of-breed open source geospatial technologies such as GDAL/OGR, PostGIS and PROJ.4
  • Integrates with popular front-end environments such as ka-Map, Chameleon, Mapbender, OpenLayers and Cartoweb

OpenLayers

OpenLayers makes it easy to put a dynamic map in any web page. It can display map tiles and markers loaded from any source. OpenLayers is completely free, Open Source JavaScript, released under the BSD License.

OpenLayers is a pure JavaScript library for displaying map data in most modern web browsers, with no server-side dependencies. OpenLayers implements a JavaScript API for building rich web-based geographic applications, similar to the Google Maps and MSN Virtual Earth APIs, with one important difference -- OpenLayers is Free Software, developed for and by the Open Source software community.

Furthermore, OpenLayers implements industry-standard methods for geographic data access, such as the Open Geospatial Consortium's Web Mapping Service (OGC WMS) and Web Feature Service (OGC WFS) protocols. Under the hood, OpenLayers is written in object-oriented JavaScript, using components from Prototype.js and the Rico library. The OpenLayers code base already has hundreds of unit tests, via the Test.AnotherWay framework.

As a framework, OpenLayers is intended to separate map tools from map data so that all the tools can operate on all the data sources. This separation breaks the proprietary silos that earlier GIS revolutions have taught civilization to avoid. The mapping revolution on the public Web should benefit from the experience of history.

Features

  • Support for a variety of data sources
  • Support for displaying geographic features, with markers and pop-ups
  • Easy build configuration, designed to help build OpenLayers into other applications
  • Javascript API to allow full control over OpenLayers-powered map from within Javascript on a web page.

Data Layer

The data layer is the foundation of any SDI. It can be built from a wide variety of spatial data formats using the GDAL/OGR library or stored in the object relational database PostgreSQL with the spatial library PostGIS. Both types of data store are supported by most web mapping server components. Additionally both GDAL/OGR and PostGIS can also be used to manipulate and convert spatial data directly.

GDAL/OGR

The Geospatial Data Abstraction Library (GDAL/OGR) is a cross platform C++ translator library for raster and vector geospatial data formats that is released under an X/MIT style Open Source license by the Open Source Geospatial Foundation. As a library, it presents a single abstract data model to the calling application for all supported formats. It also comes with a variety of useful commandline utilities for data translation and processing.

GDAL supports over 50 raster formats, and OGR over 20 vector formats.

It provides the primary data access engine for many applications including MapServer, GRASS, QGIS, and OpenEV. It is also utilized by packages such as OSSIM, Cadcorp SIS, FME, Google Earth, VTP, Thuban, ILWIS, MapGuide and ArcGIS. GDAL/OGR is the most widely used geospatial data access library.

Professional support options are available for commercial software developers desiring assistance integrating and extending GDAL, or dig in yourself and join the development team!

Features

  • Library access from Python, Java, C#, Ruby, VB6 and Perl
  • Vector data model closely aligned with OGC Simple Features
  • Coordinate system engine built on PROJ.4 and OGC Well Known Text coordinate system descriptions
  • Utilities for data translation, image warping, subsetting, and various other common tasks
  • Highly efficiency raster data access, taking advantage of tiling and overviews
  • Support for large files - larger than 4GB

PostGIS

PostGIS adds support for geographic objects to the PostgreSQL object-relational database. In effect, PostGIS "spatially enables" the PostgreSQL server, allowing it to be used as a backend spatial database for geographic information systems (GIS), much like ESRI's SDE or Oracle's Spatial extension. PostGIS follows the OpenGIS "Simple Features Specification for SQL" (OGC SFSQL) and has been certified as compliant with the "Types and Functions" profile.

PostGIS has been developed by Refractions Research as a project in Open Source spatial database technology and is released under the GNU General Public License. The roadmap includes full topology support, raster support, networks and routing, three dimensional surfaces, curves and splines and other features.

Features

  • The Open Geopstial Consortium (OGC) standards for "Simple Features" (2D Geometries)
  • The Open GIS Consortium (OGC) markup language for "Simple Features" known as Well Known Text (WKT) and Well Known Binary (WKB) representations of data.
  • OGC SQL functions: standards for managing spatial tables using SQL language
  • PostGIS extends support for geometry types EWKB, EWKT and Canonical Forms
  • PostGIS adds support for embedding the SRID (projection information) and for 3 and 4D geometries with Z and M values, respectively.
  • Other Types, Progress on Raster Support, Proposed X3D types: TIN and COLLADA for Web Output
  • PostGIS enables powerful spatial analysis and editing of geometries
  • PostGIS can be extended by scripting with an array of languages
  • Client-side extensions exist to access spatial data in PostgreSQL for the programming languages, Python, Java (own class model, JTS, read-only support for Java2D, PHP and Ruby

Architecture Perspectives

The architecture of any SDI can be depicted in simple diagrams with three sections, the data store, the service and the client level. The diagram below additionally differentiates between the geospatial services (software) and the interfaces (standards) as developed and defined by the OGC and normed by ISO. Taking simplicity to be the prime goal of any general architecture design is a good idea because complexity emerges by itself.


Image source, slide one with friendly permission by Jeroen Ticheler: http://geonetwork-opensource.org/download/SDI-Architecture.ppt/view

To make it short, SDIs run on the Web. It's application protocol is HTTP hiding the underlying transport layers. Simple enough. But there is an ongoing discussion around the buzz words SOA, ROA, REST, RESTful and SOAP that need to be put into context to make sense, this time having clarity in mind as a prime goal.

The messaging protocol SOAP exposes arbitrary non-uniform software procedures through messaging between service endpoints. This allows to connect software procedures via the global network by piggy packing messages wrapped within a separate protocol on the existing application protocol HTTP ignoring most of the latter protocol's potential. If this sounds over complicated that is probably right. The SOAP technology is frequently associated with the highly indistinct an nebulous service-oriented architecture paradigm (SOA) although they have nothing in common but three letters. Adding to the confusion, the way the Web works has been codified in the REST paradigm (Representational State Transfer) which gave birth to another somewhat vague paradigm of the Resource-oriented Architecture (ROA).

Both architecture paradigms use common terms but with distinctly different and frequently incompatible meanings. The same terms are used in both contexts but with different connotations and from different levels of perspective. In the ROA all resources can also be a service thus in one way making the SOA a subset of the ROA. From another perspective the ROA is an implementation of the broader SOA concept and its largest existing network is the Web.

The difference in the paradigms are that they look at the architecture from a usability point of view whereas what is commonly known as the SOA seems to have evolved from a software perspective. Potentially oversimplifying things again it could be stated that the ROA is the system architect's view that can be implemented by the software architect using Service-oriented architecture software. Luckily in most cases the software in question can be used following either paradigm, in the worst case by putting a facade in front of it that connects to the Web which REST does by itself.

Case Studies

There is a comprehensive list of links pointing to sites with case studies on the OSGeo Wiki web site:

The regular annual OSGeo Journal also contains case studies from all walks of life:

Rhineland Palatinate SDI

As an example for an Open Source based spatial data infrastructure this paper will present and discuss the SDI of the federal state of Rhineland Palatinate, Germany:

Problem Description

The state of Rhineland Palatinate operates a growing number of spatial data services with software from several vendors and needs to provide access to a broad range of users from the general public to specific secure environments. Additional spatial data related to the region is maintained through services operated by the state's municipalities, the German state and private data providers. Access to some data sources has to be secured, controlled and logged on a user, group and functional level.

The need for domain specific web based applications is growing quickly requiring comprehensive tracking and monitoring of services.

Requirements

From the problem definition the following (shortened) requirements for the implementation of the geoportal were identified. The full list of requirements is available through the Ministry of the Interior.

  • Manage any number of OGC WMS and WFS services running on an arbitrary number of different vendor software
  • Centralized spatial web service repository
  • Service meta data monitoring and custodian notification
  • Notification of Web application operator in case of changes of service meta data
  • Service availability monitor, notification of custodian / publisher / operator
  • Secure access to services that contain privacy related or restrictively licensed data
  • Binding 2500+ identities from within public administration authentication system to geoportal authorization and OWS Security Access façade
  • Transparent user management for 1000+ users outside of the public administration authentication system and bind to secure OWS access façade
  • Provide access management on a group, user, feature and functionality/application basis
  • Fully mandate capable user management
  • Meta data publishing conforming with INSPIRE

Solution

Most requirements defined by the GeoPortal.rlp analysis document could be addressed by standard out of the box software installations. The core software of the geoportal is the Mapbender framework. Some extensions had to be implemented to the meta data management layer, OWS security proxy and WFS-T service repository. All development that could be implemented as generic modules was added to the Mapbender core allowing it to be reused by other clients.

The core software stack of the geoportal installation is completely based on Open Source software:

  • Mapbender
  • MapServer
  • GeoServer
  • PostgreSQL / PostGIS
  • Typo3
  • MediaWiki

The geoportal is designed to act as a broker between users and providers of spatial data, information and spatially related services. The portal offers centralized access to spatial data in Rhineland-Palatinate through services organized by meta data in a repository.

GeoPortal.rlp serves federal state agencies, municipal authorities and private data providers as a platform to maintain their meta data and present their data and services. Direct access to the distributed data sources of each spatial product and service provider ensures that information made available by these institutions on a joint platform is up-to-date. The geoportal does not store any data but only its meta data, service meta data, user identity information and application configuration.

The Portal - User Perspective

Within the Spatial Data Infrastructure of Rhineland-Palatinate, the GeoPortal.rlp assumes the role of the service-oriented agent brokering spatial data between users and providers.

The core functionality of the portal provided to anonymous end user follows three steps:

  1. Search spatial data
  2. Select result
  3. Show map

The distributed access paradigm keeps the responsibility for the data, service, actuality and authorization with the body in charge (creator, custodian, reseller) guaranteeing a high level of topicality and actuality. The available panoply of spatial data made enables authenticated and authorized users to create their own individual spatial data product and web application. The consistent and comprehensive application of recognized ISO and OGC standards throughout GeoPortal.rlp allow for seamless integration of maps and data from different services.

Conclusion

The broader Open Source community has over years created a comprehensive stack of software that is used globally to implement spatial data infrastructures. There is a wide selection of components for a wide variety of domains and solutions, implemented in a variety of programming languages and adhering to all major standards. Well defined governance rules as set forth by the Open Source Geospatial Foundation provide for a secure environment for users and developers alike. As a legal entity OSGeo also gives the software projects themselves a legal backing that is also required by businesses who want to provide services for the components of the SDI stack or other software in the Open Source realm. There are no reasons not to use Open Source software and many reasons to do so. But the most convincing arguments can only be found out by actually using them, the entry barrier is very low and the reward high.


References