The INSPIRE Conference 2017 is approaching fast, and we are preparing our contribution to the conference this year. Together with five INSPIRE GIS partners, we support the conference through a Gold Cooperation.

This year’s motto is “INSPIRE Out of the Box”. The conference programme shows that implementation is progressing, and that many organisations actively shape what INSPIRE will be used for in the future. The Fitness of Purpose discussion and ideas on how to reduce the implementation burden also form a significant part of the talks.

So, you will be able to choose from a wide range of topics dedicated to INSPIRE implementation. From interesting panel discussions to valuable case studies, we recommend the following talks with contributions from the wetransform team:

Tuesday, 05.09.2017

Wednesday, 06.09.2017

Thursday, 07.09.2017

Friday, 08.09.2017

The INSPIRE Conference 2017 is approaching fast, and we are preparing our contribution to the conference this year. Together with five INSPIRE GIS partners, we support the conference through a Gold Cooperation.

This year’s motto is “INSPIRE Out of the Box”. The conference programme shows that implementation is progressing, and that many organisations actively shape what INSPIRE will be used for in the future. The Fitness of Purpose discussion and ideas on how to reduce the implementation burden also form a significant part of the talks.

So, you will be able to choose from a wide range of topics dedicated to INSPIRE implementation. From interesting panel discussions to valuable case studies, we recommend the following talks with contributions from the wetransform team:

Tuesday, 05.09.2017

Wednesday, 06.09.2017

Thursday, 07.09.2017

Friday, 08.09.2017

When we started working on our business plan, we collected as much data as we could about the cost of implementing and maintaining spatial data infrastructures. We used data from our own projects, from a survey conducted via the SMESpire project, from tender databases and from existing publications to find out what drives costs in implementation and the maintenance of INSPIRE services.

Now, we would like to share what options you have to make provision and usage of INSPIRE services accessible and affordable. Based on the data we’ve collected over three years, we’ve built a Total Cost of Ownership calculator and integrated it with the INSPIRE GIS pricing tool. It only requires a haleconnect.com account to login.

The methodology we use calculates the “system unit cost” based on the amount of data and the types of services to be deployed. We take typical costs for custom integration of existing infrastructure into account as well. Finally, we include the costs of maintenance and keeping up to date with new versions of INSPIRE and other standards.

Initial feedback to the Total Cost of Ownership calculator has been highly positive, so we decided to share this tool with the community:

Calculate INSPIRE GIS fees and compare Total Cost of Ownership of other solutions!

When we started working on our business plan, we collected as much data as we could about the cost of implementing and maintaining spatial data infrastructures. We used data from our own projects, from a survey conducted via the SMESpire project, from tender databases and from existing publications to find out what drives costs in implementation and the maintenance of INSPIRE services.

Now, we would like to share what options you have to make provision and usage of INSPIRE services accessible and affordable. Based on the data we’ve collected over three years, we’ve built a Total Cost of Ownership calculator and integrated it with the INSPIRE GIS pricing tool. It only requires a haleconnect.com account to login.

The methodology we use calculates the “system unit cost” based on the amount of data and the types of services to be deployed. We take typical costs for custom integration of existing infrastructure into account as well. Finally, we include the costs of maintenance and keeping up to date with new versions of INSPIRE and other standards.

Initial feedback to the Total Cost of Ownership calculator has been highly positive, so we decided to share this tool with the community:

Calculate INSPIRE GIS fees and compare Total Cost of Ownership of other solutions!

Over the course of the last 15 years, Germany developed its own set of GML-based spatial data exchange standards, known as ALKIS-AFIS-ATKIS (or short 3A NAS). Surveying organisations in all states have implemented the standards, thus providing a common foundation for an INSPIRE implementation.

In 2016, the Arbeitsgemeinschaft der Vermessungsverwaltungen der Länder der Bundesrepublik Deutschland (AdV) commissioned wetransform to create a formal data transformation documentation, with 3A NAS and the “Hauskoordinaten” as a source and with 12 INSPIRE Annex I GML schemas as the target. This documentation was to be generated based on hale studio alignments, and validated against data sets from multiple German states.

This project has recently been completed, resulting with the first full, formal and executable data transformation specification. Those results helped German authorities to achieve the breakthrough in the provision of harmonised INSPIRE data sets.

Additional project challenge was the high complexity of the source data models, being much larger than the individual INSPIRE annex data specifications. Furthermore, the 3A NAS schemas use a lot of special constructs to link features, and the German states have implemented individual variants due to different processes and legal requirements.

In this article, we explain how we created these highly complex alignments with up to 450 cells using hale studio, what methodologies we applied, and how implementers, e.g. the state of Rheinland-Pfalz have already picked up the results to create INSPIRE compliant data sets from their 3A NAS production databases.

The baseline for the project was a massive collection of Excel matching tables, equivalent to more than 200 A3 pages when printed out. We used these Excel tables to create the initial Alignments. Furthermore, we worked with the AdV to define common rules for the transformation and for the resulting INSPIRE data sets, such as patterns for gml:id and gml:identifier elements.

Over the course of the last 15 years, Germany developed its own set of GML-based spatial data exchange standards, known as ALKIS-AFIS-ATKIS (or short 3A NAS). Surveying organisations in all states have implemented the standards, thus providing a common foundation for an INSPIRE implementation.

In 2016, the Arbeitsgemeinschaft der Vermessungsverwaltungen der Länder der Bundesrepublik Deutschland (AdV) commissioned wetransform to create a formal data transformation documentation, with 3A NAS and the “Hauskoordinaten” as a source and with 12 INSPIRE Annex I GML schemas as the target. This documentation was to be generated based on hale studio alignments, and validated against data sets from multiple German states.

This project has recently been completed, resulting with the first full, formal and executable data transformation specification. Those results helped German authorities to achieve the breakthrough in the provision of harmonised INSPIRE data sets.

Additional project challenge was the high complexity of the source data models, being much larger than the individual INSPIRE annex data specifications. Furthermore, the 3A NAS schemas use a lot of special constructs to link features, and the German states have implemented individual variants due to different processes and legal requirements.

In this article, we explain how we created these highly complex alignments with up to 450 cells using hale studio, what methodologies we applied, and how implementers, e.g. the state of Rheinland-Pfalz have already picked up the results to create INSPIRE compliant data sets from their 3A NAS production databases.

The baseline for the project was a massive collection of Excel matching tables, equivalent to more than 200 A3 pages when printed out. We used these Excel tables to create the initial Alignments. Furthermore, we worked with the AdV to define common rules for the transformation and for the resulting INSPIRE data sets, such as patterns for gml:id and gml:identifier elements.

Base Alignments and Custom Functions

During the initial analysis of the data models, we saw the need for specific functions and common mappings for all the alignments. As both, the source and target models are rich object-oriented models with rich inheritance hierarchies, we can define the common mappings in one alignment and then import these into all others. These so-called base alignments are re-usable components that we then imported into all Annex I alignments:

• base-functions: Common functions for all themes (extended also by the other base alignments)
• base-tn: Common functions and mappings for rail transport, road transport, water transport, air transport, cable transport
• au-basis: Common mappings for all variants of the Administrative Units Alignment

The custom functions we wrote for this project included the following:

• Creation of Geographical Name objects
• Specific simplification rules for geometries
• Generation of local IDs and Identifiers according to the AdV identifier rules
• Conversion of units of measurement

Using the custom functions, we avoided a lot of redundancy in the alignments and reduced their complexity.

The Annex I Alignments

The core task in the project was to create the 14 concrete alignments used to generate the formal documentation. We applied the following development process:

1. Implement the alignment according to the provided mapping tables, collecting any ambiguous points and posting questions on the project issue tracker
2. Provide the alignments, the generated documents and the transformed data for review to the AdV stakeholders
3. Implement improvements and fixes as suggested

In this project, we learned that the highly detailed matching tables captured only about 30% of all transformation cells in the final projects correctly or fully. Most of the work was to review and improve iterations that followed on the initial implementation. A lot of very important input was provided by the AdV stakeholders, so that the alignments could be improved until they reached sufficient quality on all aspects. The following links lead you to the interactive mapping documentation for some of these:

1. Hauskoordinaten to INSPIRE Addresses
2. 3A to INSPIRE Addresses
3. 3A Flurstücke to INSPIRE Cadastral Parcels
4. 3A Flurstücke to INSPIRE Administrative Units
5. 3A Gebiete to INSPIRE Administrative Units
6. 3A kommunale Gebiete to INSPIRE Administrative Units
7. 3A to INSPIRE Air Transport Network
8. 3A to INSPIRE CableTransport Network
9. 3A to INSPIRE Road Transport Network
10. 3A to INSPIRE Railway Transport Network
11. 3A to INSPIRE Water Transport Network
12. 3A to Geographical Names
13. 3A to Hydro-Physical Waters
14. 3A to Hydrography Network

These alignments are currently in the final resolution process of the AdV.

Variants and Derived Alignments

You might have noticed that there are three alignments that have Administrative Units as their target schema: In 3A, the geometry of Administrative Units is derived by creating the union of a set of land parcels. This process reduces redundancy in the data, but can be computationally expensive. As a consequence, we developed an alignment that creates these aggregated geometries for all levels of Administrative Units, but also made two variants that allow the specification of an additional data source with the respective pre-aggregated geometries.

We set up a process to generate derived alignments for subsets of the 3A data models based on the “Modellart”. The “Modellart” is a mix of model and scale – for example, there are landscape models in scales of 1:25.000 to 1:1.000.000. Each “Modellart” includes a subset of the total 3A model, so that the transformation also need to be used on a subset only, and some information is not available. We used annotations to the mapping cells to indicate which cell is relevant for which model. Due to hale’s declarative mapping they can be created easily by excluding mappings for feature types that are not part of the respective model.

We also set up another automated generation process to derive modified alignments that would use the PostNAS database system instead of 3A XML as the source schema. One of the big advantages of a declarative system is that it makes such derivation processes and re-used of transformation mappings feasible.

Continuous Testing

For any kind of complex data processing, continuous testing is necessary. We set up an automated process that transformed and validated more than a dozen different data sets after each change to the mappings. This process was implemented with a Gradle script invoking the hale Command Line Interface. This interface has grown in capabilities with each release and can be used to control almost all aspects of hale – be it the transformation, the generation of artifacts such as the formal documentation or the validation of the results.

Documentation

The final deliverable of the project was the formal documentation. For a long time, hale studio had the capability to generate both matching tables and HTML documentation. Over the development of the last releases we have continuously improved the HTML documentation feature, so that the documentation offers a lot more than any static document could provide. It includes a graphical representation of the mapping, a verbal description, and information on the related schema entities, notes and other information. It is also interactive –search and filter options make it possible to choose what information to display.

Collaboration

This project was a relatively complex undertaking, with more than 20 stakeholders reviewing the mappings and the transformed data to ensure completeness and correctness of the formal documentation. In the initial project, we used Gitlab as an issue tracker and collaboration platform. Gitlab is a very useful general purpose project and source code management platform, much like GitHub. However, we also found some issues with the usage of Gitlab for this specific use case:

• Due to the size of the alignments, it was hard to establish context (e.g. which cell, which data) for any reported issues; reviewers used screenshots of the documentation
• It was hard to keep track of changes made to the same mapping cells, so that repetitive and competing solutions were implemented
• Standard diffs don’t work well to communicate changes and the history to a mapping cell to the domain experts involved in the projects

We thus implemented collaboration features as part of the documentation itself. These collaboration features enable efficient teamwork in larger groups with diverse backgrounds:

• Tasks: Create and assign tasks to users of the platform if something needs to be changed in the transformation project.
• Comments: Start a discussion visible to anybody with access to the transformation project in scope of a single mapping function or in scope of the entire alignment.
• Notes: Add a private note for yourself to the alignment or any single cell.

These additional features require a central service to function, which we deployed as part of haleconnect.com. We evaluated the use of hale connect to manage our internal transformation projects over the last months. Now, we start to use the same processes with our customers to build better transformation projects faster.

On June 20th and 21st, Eurogeographics, Geonovum and EuroSDR organized a workshop on Extensions for INSPIRE Data Specifications. Over 40 participants gave a grand total of 26 presentations! In this article, I will highlight some of the aspects that I found particularly interesting.

Before we delve into each of the individual presentations and discussions, I’d like to point you to a video the JRC has just released that provides a good intro into the “why” and “how” of INSPIRE Data Specifications:

On June 20th and 21st, Eurogeographics, Geonovum and EuroSDR organized a workshop on Extensions for INSPIRE Data Specifications. Over 40 participants gave a grand total of 26 presentations! In this article, I will highlight some of the aspects that I found particularly interesting.

Before we delve into each of the individual presentations and discussions, I’d like to point you to a video the JRC has just released that provides a good intro into the “why” and “how” of INSPIRE Data Specifications:

After the introductions to the topic by Morten Borrebaek (Norwegian Mapping Authority), Dominique Laurent (IGN France) and Michael Lutz (JRC), we looked into the first examples for data specifications related to INSPIRE.

Giacomo Martirano explained the work he has been doing for a Damage and Loss Data Model based on the Natural Risk Zones data specification. The key components from the INSPIRE data specification which he re-used included the Geometry model and the Identifier model, and decided to use the Association pattern to link INSPIRE reference data with the types from the new Damage and Loss Data Model.

Heidi Vanparys presented the Danish geodata model. I was particularly interested in their approach to flattening and simplification, which had been discussed in the MIG before as well. Their group followed the ISO methodology since 2013 and collected a lot of experience. They decided to use separate national modelling principles for their data models to enable them to be more agile with changes, and used the INSPIRE data specifications more as a source of inspiration. As an example, Heidi mentioned the use of DKFeatureType as the base type definition instead of gml:featureType. She also explained the use of additional tagged values and of formalized comments, and added that they learned some key lessons:

• A common understanding of data between different domain experts and technical staff is the basis for any successful data modelling.
• Focus on the “signal” when creating a model, keep the “noise” low by limiting imports and reducing non-essential structures

Ad van Houtum from Kadaster continued with an impressive presentation on how business requirements drove the development of an extension to the INSPIRE utility network model. This extension for the KLIC system helps to reduce excavation damage, of which there were 32.000 cases per year with an average damage 850 € per case. His group worked with Geonovum and used the Mix-In/Multiple Inheritance pattern to be compliant to both INSPIRE and national models on a conceptual level.

On we went with the presentation by Knut Jetlund on the Norwegian Road Transport model. They defined a conceptual relationship between the INSPIRE types and their own model, using the UML Realize pattern (not yet added to the pattern catalogue). Compared to the inheritance pattern which mandates that a subtype is fully compatible to the subtype, Realize and Redefine work differently and allow more freedom in the actual implementation. Knut used this freedom to simplify complex properties defined in the INSPIRE data models to simple properties. He explained that they created matching tables to map the simplified structures to the original ones – this pattern of “extension by mapping” is something that I’ve noticed in quite a few cases.

Another interesting approach was brought forward by Paul Janssen form Geonovum. He explained how their group created a formal annotation model for legal texts and aligned that model with INSPIRE data specifications. Similar to what Knut had presented, this alignment between the legal text model and the structures in INSPIRE is a kind of “extension/realisation relationship”.

Dominique Laurent from IGN France did something out of the trend: She presented how IGN “pumped up” the INSPIRE model, instead of flattening or simplifying it. This was necessary to capture all user requirements embodied in the Batiment Digital UNI v2 product. An interesting technical aspect of the “pumping” was the usage of JSON objects in database fields. Dominique pointed out that these need appropriate editors. Again, no formal Extension pattern was used – INSPIRE served mostly as inspiration. Dominique explained that the new data model was designed so that transformation to other models would be easier, and that this “extension by mapping” was documented in matching tables. IGN also decided to work with Excel tables instead of UML as a modelling tool. The team did not see UML as useful, as their model has no inheritance, doesn’t have many associations, and a graphical presentation was not required.

A bit later, the first day concluded with discussions around lessons learned, methodologies to use, and the question of whether an extensions registry would be helpful to the community. After the much deserved cold beer and dinner, we all went to bed, and most of us joined on the second day to continue the workshop.

Morten Borrebaek presented how his team built on the INSPIRE Planned Land use theme, with the objective of aligning the national standard SOSI and INSPIRE in one model. Their approach is entirely model driven, using UML models for SOSI standards, and mapping to realization platforms such as GML or databases. The usage of the Realization pattern allowed them to alias all element names to Norwegian. This work is part of a national strategy, and all related standards are documented at geonorge.no.

Katalin Toth (JRC) then gave a presentation on the EU’s Common Agriculture Policy (CAP), which opened my eyes on the potential of an infrastructure like INSPIRE; but it also showed how difficult it can be to resolve the political problems. Technical issues appear tiny in comparison!

Helen Eriksson explored the differences between SWSS and INSPIRE Hydro-Physical waters and set herself the challenge to build really complex extensions. She largely followed the ELF modelling principles and the methodology documented at the INSPIRE extensions project site. She was also one of five presenters using hale studio to perform the transformation of existing data to the new extension.

Before we went into the lunch break, the participants discussed on required research topics, such as alternative access patterns like APIs, Apps and Solutions, but also about Organisation and Policy.

After the lunch break, it was time for my session – which was announced as a “Main learnings of the Geonovum Masterclass”. I wasn’t entirely sure what to add after two days of intense technical presentations, but decided to first share some observations, help people ask the right questions and then to set them on the path to find their own way. During the masterclass, I revisited the methodology and focused on four topics:

• Documentation of the model and of its relationships to other models
• Validation based on real data and on quantified model analysis techniques
• Agile maintenance and evolution of the extension
• Tooling, such as validation frameworks and modelling tools

We concluded with short summaries given by Morten, Dominique and others. At this point I’d like to thank the organisers for this interesting opportunity to continue the work started last year with the Geonovum project, and would like to wish everyone “Happy Extending”!

Together, Spatineo and Wetransform provide integrated publishing and monitoring of spatial data through web services.

Spatineo is well known in the community to provide monitoring and analytics for spatial web services. Wetransform is mostly known for data transformation and publishing solutions for spatial data providers. Wetransform’s INSPIRE GIS platform is an easy to use solution that makes it effortless to publish, manage and update geospatial data. Keeping this data to a high standard is crucial, since INSPIRE mandates minimum uptime and defines reporting requirements. Spatineo Monitor helps achieve these goals, by constantly testing services, keeping data owners up-to-date, and automating reporting to INSPIRE.

Doesn’t that sound like a logical partnership? We thought so too - after the 2016 edition of the INSPIRE conference, we worked together to build a joint offering to provide more value to our clients.

Together, Spatineo and Wetransform provide integrated publishing and monitoring of spatial data through web services.

Spatineo is well known in the community to provide monitoring and analytics for spatial web services. Wetransform is mostly known for data transformation and publishing solutions for spatial data providers. Wetransform’s INSPIRE GIS platform is an easy to use solution that makes it effortless to publish, manage and update geospatial data. Keeping this data to a high standard is crucial, since INSPIRE mandates minimum uptime and defines reporting requirements. Spatineo Monitor helps achieve these goals, by constantly testing services, keeping data owners up-to-date, and automating reporting to INSPIRE.

Doesn’t that sound like a logical partnership? We thought so too - after the 2016 edition of the INSPIRE conference, we worked together to build a joint offering to provide more value to our clients.

This offering focuses on monitoring spatial data services created by wetransform’s INSPIRE GIS platform and on integrating statistics into the platform, but also goes beyond that. Wetransform’s founder, Thorsten Reitz, explains: “In our industry, the big players have a strong partner network that enables them to work effectively and to reach large numbers of customers. We think that this is also possible when startups and SMEs collaborate.”

Spatineo and Wetransform started integrating their platforms this spring, and now have the first customers using the integration. Spatineo Monitor and Wetransform’s INSPIRE GIS platform strengthen each other and in the near future this teamwork will be bringing more sophisticated services to our clients.

As of today, the integration provides the following functionality:

• Automated registration and updating of new services at the Spatineo Monitor
• Optional automated publishing in the Spatineo Directory
• Display of availability and performance data in INSPIRE GIS

Spatineo’s Managing Director Sampo Savolainen is excited about the cooperation: “Combining Spatineo Monitor’s quality assurance with Wetransform’s excellent INSPIRE GIS platform gives our customers a strong and reliable platform to build successful services that support the application ecosystems using the data.”