News > INSPIRE
News' image preview

Together, Spatineo and Wetransform provide integrated publishing and monitoring of spatial data through web services.

Spatineo is well known in the community to provide monitoring and analytics for spatial web services. Wetransform is mostly known for data transformation and publishing solutions for spatial data providers. Wetransform’s INSPIRE GIS platform is an easy to use solution that makes it effortless to publish, manage and update geospatial data. Keeping this data to a high standard is crucial, since INSPIRE mandates minimum uptime and defines reporting requirements. Spatineo Monitor helps achieve these goals, by constantly testing services, keeping data owners up-to-date, and automating reporting to INSPIRE.

Doesn’t that sound like a logical partnership? We thought so too - after the 2016 edition of the INSPIRE conference, we worked together to build a joint offering to provide more value to our clients.

Together, Spatineo and Wetransform provide integrated publishing and monitoring of spatial data through web services.

Spatineo is well known in the community to provide monitoring and analytics for spatial web services. Wetransform is mostly known for data transformation and publishing solutions for spatial data providers. Wetransform’s INSPIRE GIS platform is an easy to use solution that makes it effortless to publish, manage and update geospatial data. Keeping this data to a high standard is crucial, since INSPIRE mandates minimum uptime and defines reporting requirements. Spatineo Monitor helps achieve these goals, by constantly testing services, keeping data owners up-to-date, and automating reporting to INSPIRE.

Doesn’t that sound like a logical partnership? We thought so too - after the 2016 edition of the INSPIRE conference, we worked together to build a joint offering to provide more value to our clients.

Directly publish services from INSPIRE GIS to Spatineo Monitoring and Catalogues

This offering focuses on monitoring spatial data services created by wetransform’s INSPIRE GIS platform and on integrating statistics into the platform, but also goes beyond that. Wetransform’s founder, Thorsten Reitz, explains: “In our industry, the big players have a strong partner network that enables them to work effectively and to reach large numbers of customers. We think that this is also possible when startups and SMEs collaborate.”

Spatineo and Wetransform started integrating their platforms this spring, and now have the first customers using the integration. Spatineo Monitor and Wetransform’s INSPIRE GIS platform strengthen each other and in the near future this teamwork will be bringing more sophisticated services to our clients.

As of today, the integration provides the following functionality:

  • Automated registration and updating of new services at the Spatineo Monitor
  • Optional automated publishing in the Spatineo Directory
  • Display of availability and performance data in INSPIRE GIS

Spatineo’s Managing Director Sampo Savolainen is excited about the cooperation: “Combining Spatineo Monitor’s quality assurance with Wetransform’s excellent INSPIRE GIS platform gives our customers a strong and reliable platform to build successful services that support the application ecosystems using the data.”

(more)

News' image preview

INSPIRE Pilots and Data Harmonisation Case Studies

To support the INSPIRE implementation and understanding, the JRC coordinates several large-scale INSPIRE pilot projects, such as the Marine Pilot, the Transportation Pilot and also the Danube Reference Data and Services Infrastructure (DRDSI).

DRDSI is an initiative that aims to provide support for the implementation of the European Union Strategy for the Danube Region (EUSDR) in close cooperation with key scientific partners. The initiative covers a lot of use cases and datasets that are inherently cross border, such as…:

  • River Basin District Management
  • Assessment of Water Resources
  • Environmental Impact Analysis

INSPIRE Pilots and Data Harmonisation Case Studies

To support the INSPIRE implementation and understanding, the JRC coordinates several large-scale INSPIRE pilot projects, such as the Marine Pilot, the Transportation Pilot and also the Danube Reference Data and Services Infrastructure (DRDSI).

DRDSI is an initiative that aims to provide support for the implementation of the European Union Strategy for the Danube Region (EUSDR) in close cooperation with key scientific partners. The initiative covers a lot of use cases and datasets that are inherently cross border, such as…:

  • River Basin District Management
  • Assessment of Water Resources
  • Environmental Impact Analysis
Data sets being transformed to INSPIRE usign hale studio

In the scope of the DRDSI project, we implemented a data mapping and transformation pilot. The pilot involved all steps of a data harmonization project, from source data analysis over transformation to data publishing, and was conducted with a short timeframe and relatively small budget. The data harmonisation pilot was commissioned by the JRC and executed by wetransform.

Through this work, we created harmonized data that adds content and value to the existing DRDSI. The work aimed at filling in gaps in regional datasets by creating harmonised data for bordering countries and documenting results for use in the DRDSI platform.

Analysis

As a first step, we wanted to know whether the existing data is fit for INSPIRE harmonisation. We received seven data sets from Moldova and five data sets from Ukraine. For all data sets, we performed a quick quality analysis. This analysis included the following checks:

  • Completeness: Are all fields in the source data filled?
  • Consistency: Are there many inconsistent values, such as overlapping geometries, or different spellings of the same names?
  • Coverage: Can we likely get the minimum required information to fulfill INSPIRE requirements from the source data sets?
  • Encoding: Is the encoding clear and correct?

Based on the analysis, we decided to use data sets for three different INSPIRE themes for the pilot: Administrative Units, Hydro-Physical Waters and Railway Transport Networks.

Transformation

In many INSPIRE implementation projects, there are two steps: conceptual mapping and transformation development. With hale studio, both steps can be combined into one. There are several functions we used to make sure we got the mapping right – both conceptually and technically. In particular, we used hale studio’s real time validation features based on the loaded source data to assess whether our target data set is schema compliant. For review of the mapping by the data providers, we generated the interactive documentation and worked on improvements together. You can check out two example transformation projects we created here:

Publishing

When the transformation projects were completed, the next step was to publish the data as INSPIRE View and Download services.

We generally provide two options how to deliver services: Either as Docker Containers, or as public cloud services. As the data providers and research partners in the project didn’t have resources to host the services, we agreed to use haleconnect.com to publish the data sets. However, we also provided instructions on how the project partners could set up services based on degree directly.

Conclusions

The objective of this project was to quickly implement INSPIRE data sets and services to enable cross-border use cases for the Danube Reference Data and Services Infrastructure. We were able to work very effectively with the data stakeholders, who helped us with the analysis and the mapping through their profound understanding of the data. Using hale studio and hale connect, we acquired, analysed, transformed and published 6 INSPIRE data set with a total effort of about 10 person days.

(more)

News' image preview

One thing we’re doing a lot for our customers is to create INSPIRE data sets from their original data. Usually these data sets are available in a specific national or organisation-specific schema and need to be restructured substantially to meet the INSPIRE requirements. This harmonisation process is one of the things that has given INSPIRE a bad reputation, as in that it is a complex and time-intensive endeavour.

Recently, we passed the 100-datasets-harmonised mark. As we usually track the effort needed for each of these projects, we now start to have a meaningful sample size to judge how much time the development of each of these transformation projects took – time to look at some numbers!

The data that we collect for every project includes the source schema, the target schema, the time spent and a few additional variables, such as schema complexity. In this post, we’re going to look at the mean time spent per target data model, we’ll look at the correlation between source model complexity and effort as well as simple counts.

The dataset

Out of all the projects we’ve done, 68 have time tracking records, and are related to INSPIRE – either they use one of the 34 core data specifications, or an extension of one of those.

One thing we’re doing a lot for our customers is to create INSPIRE data sets from their original data. Usually these data sets are available in a specific national or organisation-specific schema and need to be restructured substantially to meet the INSPIRE requirements. This harmonisation process is one of the things that has given INSPIRE a bad reputation, as in that it is a complex and time-intensive endeavour.

Recently, we passed the 100-datasets-harmonised mark. As we usually track the effort needed for each of these projects, we now start to have a meaningful sample size to judge how much time the development of each of these transformation projects took – time to look at some numbers!

The data that we collect for every project includes the source schema, the target schema, the time spent and a few additional variables, such as schema complexity. In this post, we’re going to look at the mean time spent per target data model, we’ll look at the correlation between source model complexity and effort as well as simple counts.

The dataset

Out of all the projects we’ve done, 68 have time tracking records, and are related to INSPIRE – either they use one of the 34 core data specifications, or an extension of one of those.

Data set counts by required time for transformation project development

As the graph shows, quite exactly half of the projects can be completed in 8 hours or less, while only very few projects took more than 64 hours to complete. 64 hours equal about 10 productive person days when we factor in some overhead.

After looking at the general effort distribution, we wanted to dig a bit deeper – which INSPIRE Annex themes create a lot of effort for us?

Efforts to create a transformation project by target schema

The range the graph shows is pretty wide. While Addresses, Transport Networks and Hydrography Networks are all in the 30+ hour range, most of the other themes show mean times of 5 to 20 hours of required effort. As the orange line in the graph indicates, the number of datasets we’ve included for a given target data model is in many cases very small (1-3), so these numbers are certainly not stable.

Maybe we need to look at the dataset from a different angle. As we often work on a fixed price basis, we want to make sure the estimates we give are reliable, so it is important for us to know what drives effort up. Thus, the next thing we look at is source data model complexity. We measure complexity using an arbitrary set of measures that tests existence of some model features (such as foreign key relationships and inheritance) as well as model size to give a number between 1 (e.g. a single shapefile) and 10 (massive model, with every modelling feature you can imagine).

Effort required for transformation project development by Source Model Complexity

This graph does show an interesting – and not really unexpected relationship. On the X-Axis, we can see the source model complexity, on the Y-Axis, we see the time spent for the projects. We indicate effort and complexity for each project with a blue dot, and the trendline with an orange dotted line. The relationship is pretty clear: The more complex the model, the higher the mean effort. The trendline is actually almost linear, and shows a growth from about 3 to 28 over the complexity range from 1 to 10 – which is a factor close to 10.

Our conclusions?

  • Source model complexity is so far the best indicator for expected effort in a project;
  • Effort varies a lot across different INSPIRE themes;
  • Overall, more than half of the INSPIRE harmonisation projects can be completed in less than a day (caveat: we are quite experienced, so a person knowing less about INSPIRE and hale studio will need more time).

What are your experiences? How much time did you spend on transformation project setup?

(more)

News' image preview

Michael Lutz and Athina Trakas decided to spice up the OGC Europe forum slot at the Delft Technical Meeting by asking for position papers around the question “What if we would start implementation of INSPIRE again today?”

Michael Lutz and Athina Trakas decided to spice up the OGC Europe forum slot at the Delft Technical Meeting by asking for position papers around the question “What if we would start implementation of INSPIRE again today?”

Participants of the workshop engage in discussions, Photo by Michael Lutz

More than 40 participants joined the workshop and first of all, saw eight three-minute presentations with a wide range of suggestions:

  • Satish Sankaran (Esri Inc.) asked “What are the right metrics to measure success?”, and suggested that adoption rates could be improved if people could contribute to the infrastructure without requiring full compliance.
  • Paul van Genuchten (GeoCAT) highlighted the potential of INSPIRE linked data as explored in the GeoNovum testbed Geo4Web.
  • Thijs Brentjes (Geonovum) suggested a set of data specifications with a simple base, with no mandatory extensions, built on top of existing (INSPIRE) SDI. He also suggested to use additional encodings with the objective of making INSPIRE usable by web developers.
  • Sylvain Grellet (BRGM) also suggested that adoption would be easier with alternative encodings (SF, JSON, …). He was the first presenter to suggest different levels or labels for compliance. Sylvain also said that joint funding of development should be organised from the start on, instead of leaving everything to the implementers. Sylvain also suggested to better organise aspects such as trainings and hackathons.
  • Clemens Portele (Interactive Instruments) explained how important stability and reliability are for a major infrastructure project like INSPIRE. He suggested to improve specifications through small, iterative changes. It should be possible to make these changes in an agile, fast, usage driven way. He briefly outlined the work done by Geonovum and Interactive Instruments on making data accessible to web applications and suggested to put facades or proxies on top of the existing INSPIRE infrastructure.
  • Thorsten Reitz (wetransform) focused on the user experience of applications built directly to manage and explore INSPIRE data and services and explained that with most of the investment going to backend infrastructure, these leave a lot to be desired would be essential to show the value of the infrastructure.
  • A representative of Natural Resources Canada explained the objectives of the Maps for HTML standards working group and explained that adding capabilities such as MapML would foster usage and adoption.
  • Peter Bauman (Rasdaman GmbH) asked “What if our services could talk?” but wasn’t able to join in person.

These three recurring topics emerged:

  1. tiered or more flexible compliance,
  2. usage of web standards and improvements to data usability and
  3. adding proxy layers on top of the INSPIRE infrastructure.
Participants of the workshop engage in discussions, Photo by Michael Lutz

Following this agenda setting, we split up in four groups to discuss several key questions, using the World Cafe methodology:

  • What standards and technologies should the infrastructure be based on?
  • What architectural pattern would you recommend? What should be the main components of the infrastructure?
  • How would you organise the implementation process and make it cost-efficient?
  • How would you ensure a wide adoption and use of the infrastructure?

Athina and Michael asked me to facilitate the group discussion around the third question - “How would you organise the implementation process and make it cost-efficient?” Our objective was to define 2-3 recommendations and to suggest follow-up actions. As facilitator, I asked the following questions, and a lively discussion ensued:

What is the very first thing that should happen in the implementation process?

  • Define criteria for success early on (not “only” for compliance)
  • Define Interoperability (when is data interoperable -> when it us useable in clients), “General” interoperability isn’t the goal, usage is!
  • Provide end product specifications for high value use cases to large numbers of users
  • Define success for concrete users in addition to the overall objective

If you had one year to implement INSPIRE from scratch, how would you do it?

  • Start with simple data models, manage complexity (necessary vs. unnecessary, e.g. in ISO and metadata), expand over time with new use cases
  • Follow pragmatic approaches
  • Use a well-defined set of interfaces that are already proven (mainstream IT industry)
  • Follow trend in industry towards RESTful mechanisms
  • Focus on pushing data out (e.g. as done in Copernicus)

What is the process to continuously coordinate implementation and make re-use possible?

  • Use agile methods such as iterative development
  • Keep users in close involvement
  • Identify anything
  • Keep existing infrastructure, build agile infrastructure on top, then let the market choose what works
  • Transparency, make it available
  • The metric could be: How many users to you have? Users drive continual improvement?

How can implementation of key components be coordinated in an efficient way?

  • Coordinate development of core components early on, in particular validation and registries (e.g. code lists, extensions)
  • Common components should be coordinated; Harmonisation may be such an issue
  • No mandatory components, but all components for a reference implementation should be available
  • Make sure that requirements that are specific to INSPIRE are really understood well - Value vs. effort/costs on very specific INSPIRE requirements?
  • Clarify the business case for the implementation coordinator - is that organisation paid for by taxes?
  • Countries see lots of liability and low central investment
  • Central funds/investments is low compared to the overall investments required for implementation and compared to COPERNICUS
  • Harmonisation across INSPIRE and Copernicus currently looks like a Godzilla vs. King Kong fight, will be difficult to achieve effectively
Participants of the workshop engage in discussions, Photo by Michael Lutz

We then consolidated recommendations:

  1. We should treat INSPIRE not as separate infrastructure, but rather as integrated with existing products and processes, e.g. by extending national models to meet additional INSPIRE requirements
  2. Cost effective: INSPIRE not as something specific, but as a general infrastructure and be a natural part of what we are doing
  3. We would reframe INSPIRE in the context of the Open Data Movement to limit competition between INSPIRE and Open Data / Linked Open Data
  4. We also make sure products are designed from the user experience first
  5. We have to orient implementation guidance towards implementer’s questions and problems (“How to provide a bridge in INSPIRE” –> can’t be answered by a professional)
  6. Have a library of reference implementations to describe how it’s done for all annexes

… and Follow-Up actions:

  1. Collect reference implementations as concrete guides and publish those
  2. Provide compliance levels (?) as a means to get in easily
  3. Find how to react to new uses cases in an agile way

A side discussion that came up in our group as well as in at least two of the other three groups was what it meant to be compliant really, but I’ll leave that for another post ☺.

All in all, I really enjoyed the highly interactive format of the What if…? Workshop and the productive discussions, which was not just rehashing of previously discussed issues. Thanks to Athina and Michael!

(more)

News' image preview

The central event for all people involved in the design and implementation of the INSPIRE infrastructure is coming up! From 26th to 30th of September, more than 900 people will meet in Barcelona to learn the latest about best practices, technologies, costs and benefits. Want to know what wetransform and the INSPIRE GIS partners have been up to? Join one of our five presentations, the workshop or visit our booth in the exhibition hall. Through our Silver sponsorship of the conference, we support the community and look forward to a great meeting!

The central event for all people involved in the design and implementation of the INSPIRE infrastructure is coming up! From 26th to 30th of September, more than 900 people will meet in Barcelona to learn the latest about best practices, technologies, costs and benefits. Want to know what wetransform and the INSPIRE GIS partners have been up to? Join one of our five presentations, the workshop or visit our booth in the exhibition hall. Through our Silver sponsorship of the conference, we support the community and look forward to a great meeting!

Barcelona Conference Center

These are our workshops and presentations:

Workshop: INSPIRE Extensions: Guidance for building data models on the Data Specifications (Thorsten Reitz, Jandirk Bulens, Stefania Morrone, Kathi Schleidt)

Monday, 26.09.2016, 09:00 to 12:30, Rooms B1-B2

In this workshop, we present results of our survey on data modelling and an inventory of existing extensions. We then present a common baseline methodology and a pattern catalogue for relating new data models with INSPIRE. In addition, we invited participants in the survey to submit a presentation to the INSPIRE conference. We provide slots for two of these presentations in the workshop, to complement what participants learned with real-world experiences from their peers.

More information on the workshop is available at the inspire extensions website.


INSPIRE as a Service - Ensuring Compliance and Availability (Andreas von Dömming, Thorsten Reitz)

Tuesday, 27.09.2016, 15:00 to 15:30, Room B1-B2

In this talk, we will explain how an INSPIRE as a Service platform based on public cloud technology can fulfill legal compliance rules, such as ISO27001 or the German BSI Grundschutz. We will also discuss the INSPIRE Service Level requirements on availability and throughput. We furthermore highlight options to validate metadata, data and services.


Reducing implementation and maintenance costs through INSPIRE as a Service (Thorsten Reitz)

Wednesday, 28.09.2016, 09:15 to 09:30, Room A

In this talk, we present an in-depth analysis of contributors to costs in implementation and in the maintenance of INSPIRE services, and look at strategies to reduce these costs substantially.

Our goal is to show what options the community has to make provision and usage of INSPIRE services accessible to all data providers and users. As a basis for the analysis, we use data from our own projects, from tendering databases and from existing publications. In the cost analysis, we look at the components that lead to the total system unit cost. One such component is the service type and usage, a second one is the license type. We also include a comparison of different service types and their operational requirements, factoring in INSPIRE requirements on availability and capacity.

We finally present scenarios where INSPIRE as a Service systems are shared by multiple organisations - by tens, by hundreds, and by thousands - to reduce both service provision unit costs and costs associated with using INSPIRE services.

More information on the workshop is available at the INSPIRE conference website.


How to extend INSPIRE to create added value? (Jandirk Bulens, Thorsten Reitz)

Wednesday, 28.09.2016, 09:30 to 09:45, Room H2

In this talk, we present an overview of current best practices to create and use extensions on INSPIRE. We outline patterns to be used by data providers in extending INSPIRE in the harmonisation process. The talk covers the inventory carried out of model extensions, an INSPIRE based Model-Driven approach, the Extension Methodology, the resulting Pattern Catalogue and an End-to-End Tutorial Project. We share the conclusions of the study and give recommendations for follow up activities.

DG ENV, EEA and JRC support this study. It is done with the help of a European expert group providing feedback based on own experience and expertise. This talk is related to the workshop ‘INSPIRE Extensions: Guidance for building data models on the Data Specifications.’

More information on the workshop is available at the INSPIRE conference website.


INSPIRE as a Service - Clients and Business Processes (Dirk Frigne, Thorsten Reitz)

Wednesday, 28.09.2016, 09:30 to 10:00, Room B3

OK, so now that we’ve got the INSPIRE services, how can we use them to support existing or new business process and applications in an effective manner? In this talk, we’ll use the example of SpotBooking to show how INSPIRE data can reduce cost and support high-value business processes.


GDI-Südhessen: How hundreds of administrative and municipal districts stop worrying about INSPIRE (Thorsten Reitz, Andreas von Dömming and Martin Domeyer):

Wednesday, 28.09.2016, 16:00 to 16:15, Room A

This talk will help local and regional governments or other groups implement effective, INSPIRE compliant Spatial Data Infrastructures using cloud and Open Source on premise solutions. We focus specifically on Download and Transformation services to show how to make INSPIRE services usable in standard tools, such as web or desktop GIS, which often do not support complex GML.

In this presentation, we describe how an interest group made of municipalities and counties joined forces to implement an easy to use, cost effective INSPIRE platform for their members. A key decision the group made was to improve collaboration across the members of the interest group via the new platform. For this purpose, the group designed a set of local harmonized data models, from which the system should derive INSPIRE data and services automatically. Another core requirement for the platform was to automate metadata entry to the highest possible degree.

In the talk, you will learn how we use the an “INSPIRE as a Service” platform to publish more than 1.000 INSPIRE data sets. We will also discuss the following aspects of the resulting system in detail:

  • Design and User Experience
  • Scalability and resilience
  • Operations (Hosting and Application management)

More information on the workshop is available at the INSPIRE conference website.

(more)