News > tutorial
News' image preview

INSPIRE is not the only open standard for spatial information in Europe, obviously. In Germany, a new standard has been made mandatory in several states in the past year. This standard is for all kinds of planning data, ranging from regional plans to land development plans. The standard is called XPlanung, with the current version of the encoding being called XPlanGML 5.1, and it is mandatory for all companies submitting plans, and for all municipalities publishing plans.

There are several challenges when creating compliant plans, among them the fact that many plans are still only available as scans in PDF, TIFF or PNG format. Actual geodata is then usually limited to a few attributes and a polygon that describes the perimeter of the plan area.

One approach that the community around XPlanung has discussed in detail is to align INSPIRE Planned and Existing Land Use to XPlanung (and vice versa). This alignment was described in the form of a 218-page document. On that base, when you implement one, you can get the other standard for free. However, the length of that document already gives an indication about the complexity of the transformation. This complexity stems both from the size of the XPlanung standard, which covers all types of spatial plans, and from the numerous codelists employed in the source and target models.

For this article, we will only investigate how to map the most common type of plan, a land development plan, to INSPIRE Planned Land Use. In XPlanung, such plans are called BP_Plan. One BP_Plan can have multiple areas of validity, each of which is stored in an object called BP_Bereich.

INSPIRE is not the only open standard for spatial information in Europe, obviously. In Germany, a new standard has been made mandatory in several states in the past year. This standard is for all kinds of planning data, ranging from regional plans to land development plans. The standard is called XPlanung, with the current version of the encoding being called XPlanGML 5.1, and it is mandatory for all companies submitting plans, and for all municipalities publishing plans.

There are several challenges when creating compliant plans, among them the fact that many plans are still only available as scans in PDF, TIFF or PNG format. Actual geodata is then usually limited to a few attributes and a polygon that describes the perimeter of the plan area.

One approach that the community around XPlanung has discussed in detail is to align INSPIRE Planned and Existing Land Use to XPlanung (and vice versa). This alignment was described in the form of a 218-page document. On that base, when you implement one, you can get the other standard for free. However, the length of that document already gives an indication about the complexity of the transformation. This complexity stems both from the size of the XPlanung standard, which covers all types of spatial plans, and from the numerous codelists employed in the source and target models.

For this article, we will only investigate how to map the most common type of plan, a land development plan, to INSPIRE Planned Land Use. In XPlanung, such plans are called BP_Plan. One BP_Plan can have multiple areas of validity, each of which is stored in an object called BP_Bereich.

Type relationships to map XPlanung to INSPIRE

The basic structure in the INSPIRE model is easy enough: There is a SpatialPlan as the main feature type that has a set of OfficialDocumentation objects linked to it that provide legal and other information. We will thus first create the SpatialPlan, and then create all the OfficialDocumentation objects.

BP_Plan and BP_Bereich joined to SpatialPlan

In this type-relationship, we join the types BP_Plan and BP_Bereich based on the following conditions:

BP_Plan.id = BP_Bereich.gehoertZuPlan.href

Please note that hale studio automatically matches href anchors to gml:ids disregarding the # if present.

After doing so, the main information we can map is the set of dates that are present in the source type. There are 13 different date values available in BP_Plan, which we all map to ordinance. For this purpose, we create an instance context for each of the source dates, such as anderungenBisDatum. We then simply rename the source date to ordinanceDate, and assign the name of the source attribute as the ordinanceReference.

We also add references to different types of officialDocuments. In particular, if the source data has a rasterBasis, texte and/or begruendungsTexte, we point to that by renaming that value to officialDocument.href. The other mappings are straightforward. We directly re-use the gml:id from the source for the target, as well as an INSPIRE localId. name becomes officialTitle, rechtsstand is reclassified to processStepGeneral, and nummer becomes alternativeTitle.

BP_Plan Groovy-retyped to OfficialDocumentation

With the main SpatialPlan object having been created, we now need to create OfficialDocumentation objects for various cases. The first is that BP_Plan can contain multiple externeReferenz references. For each of these, we create a OfficialDocumentation object. To do that, we use a Groovy Retype with this small script:



What this script does is to emit an object that contains an ID created in a reproducible manner - using the URL of the reference as input string to generate a UUID.

We then assign the remaining required values to the created objects, such as the INSPIRE Identifier namespace.

BP_Bereich retyped to OfficialDocumentation

The feature type BP_Bereich can also contain references to various documents. We thus retype BP_Bereich to OfficialDocumentation and use the information from BP_Bereich in a straightforward mapping:

Transform XPlanGML BP_Bereich to INSPIRE OfficialDocumentation

Throughout these mappings you will note the use of a log of Assign (Bound) functions. These assign a constant value to the target if the source attribute is present for a given transformed object.

BP_TextAbschnitt retyped to OfficialDocumentation

Finally, A BP_Plan can also contain explicit structured text objects. We also want to bring these BP_TextAbschnitt over to OfficialDocumentation and retype them as well. Much like before, the actual mapping of the properties below is straightforward and only requires Renames, Assigns and a Assign (Bound) function:

Transform XPlanGML BP_TextAbschnitt to INSPIRE OfficialDocumentation

Exporting XPlanGML and conclusion

To export the transformed INSPIRE GML, you do not need to do anything special. You can just use the normal “Export Transformed Data” option.

It is also possible to export XPlanGML from hale studio. For this, you need to create custom data export based on the generic XML writer. This is easily done in just a minute following these steps:

  1. Go to File –> Export –> Create Custom Data Export…
  2. Pick “XML (Custom Root Element)” and click “Next”
  3. Give your custom export configuration a name (without spaces, e.g. xplangml_51) and a description
  4. Choose the correct XML root element; here, this will be XPlanAuszug, and click “Next”.
  5. Choose other settings such as formatted number output, CRS handling and pretty printing, and click “Finish”. You can now export source or transformed data as XPlanGML.

The transformation project is available via haleconnect.com. You can either download it directly from the web interface or in hale studio. In both cases, you need a (free) haleconnect account.

We will continue to build out this alignment and create others for XPlanung and INSPIRE. Let us know if you have any input or would like to get engaged in this activity!

Happy transforming!

(more)

It’s Thursday, or as we at wetransform (thanks to the influence of a certain Venezuelan on the team) call it, Juernes!

In this fourth installment of the Groovy Scripting week, we will tackle another non-trivial case: Building a complete network with nodes and links when your only input data was a bunch of spaghetti lines. This actually requires three scripts. It also shows in detail how to use priorities and other advanced features.

Like the previous posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

In many projects, we did not receive a fully consistent set of Nodes and Links from customers to create a complete Network, or there were no Nodes available at all.

These three snippets extract the first and last Coordinates from each Link (LineString/Curve) and then set up the Node objects, and then create bidirectional references.

Collect Nodes and connected Segment IDs

This is a simple Groovy Retype function that uses the links as a source. The target does not matter, since no target objects are emitted. In this example, the segments are called vaarwegvakken, and the IDs of these segments are stored in the field vwk_id. This function needs to be executed first and should thus have the highest execution priority.

This is the full script:

It’s Thursday, or as we at wetransform (thanks to the influence of a certain Venezuelan on the team) call it, Juernes!

In this fourth installment of the Groovy Scripting week, we will tackle another non-trivial case: Building a complete network with nodes and links when your only input data was a bunch of spaghetti lines. This actually requires three scripts. It also shows in detail how to use priorities and other advanced features.

Like the previous posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

In many projects, we did not receive a fully consistent set of Nodes and Links from customers to create a complete Network, or there were no Nodes available at all.

These three snippets extract the first and last Coordinates from each Link (LineString/Curve) and then set up the Node objects, and then create bidirectional references.

Collect Nodes and connected Segment IDs

This is a simple Groovy Retype function that uses the links as a source. The target does not matter, since no target objects are emitted. In this example, the segments are called vaarwegvakken, and the IDs of these segments are stored in the field vwk_id. This function needs to be executed first and should thus have the highest execution priority.

This is the full script:



You can download this script here.

This snippet is used in a Groovy Create function that has the Node type as a target. This function has to be executed second and needs to have the higher execution priority.



This script also shows how to create a geometry from a coordinate, using a specific coordinate Reference system.

You can download the script here.

In the final step, we add the references from the Links to the nodes created before, so that we have a fully navigable graph in the transport network. This script is inside a normal Groovy Function that has the domain ID of source as the source property and either the startNode or endNode property of the Link as the target property. The surrounding type transformation needs to have a normal execution priority so that it runs after the two other functions.



You can download this script here.

Happy transforming!

(more)

This is the third time to get “Groovy” this week! Today, we will look into how we can simplify Multi-Geometries to simple geometries using a specific script. This is often necessary when the target data model only allows simple geometries, but we can actually have composite and multi-geometries (MultiLineString, MultiPolygon) in the source data.

Like the previous posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Wednesday’s Script: Create LineStrings from MultiLineStrings (Johanna)

This Groovy Script was developed to aggregate MultiLineString geometries and convert them to LineStrings.

Background: This script was developed to process the 2018 HY-P Swisstopo data. A merge on the feature type level necessitated the use of aggregation to obtain one LineString per feature. The script below was used in a Greedy Groovy script function on the geometry property.

This script also contains a lot of great logging examples!

This is the full script:

This is the third time to get “Groovy” this week! Today, we will look into how we can simplify Multi-Geometries to simple geometries using a specific script. This is often necessary when the target data model only allows simple geometries, but we can actually have composite and multi-geometries (MultiLineString, MultiPolygon) in the source data.

Like the previous posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Wednesday’s Script: Create LineStrings from MultiLineStrings (Johanna)

This Groovy Script was developed to aggregate MultiLineString geometries and convert them to LineStrings.

Background: This script was developed to process the 2018 HY-P Swisstopo data. A merge on the feature type level necessitated the use of aggregation to obtain one LineString per feature. The script below was used in a Greedy Groovy script function on the geometry property.

This script also contains a lot of great logging examples!

This is the full script:



You can download this script here and import it in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. It assumes a certain geometry attribute name (the_geom) that you might have to change.

Happy transforming!

(more)

This is the second installment of our “Groovy week”. This time, we are going to look into a powerful feature in hale studio that is used by the Spatial Join function, but can also be accessed through scripting - the spatial index.

Like the previous post, note that this article assumes you have working knowledge of hale studio and know the terminology.

Tuesday’s Script: Build Polygons from Lines (Florian)

We used this script to build up references from boundaries to four levels of administrative units. In the original data, such references were not explicit, with the exception of international borders.

The script first identifies possible candidate AdministrativeUnit polygon geometries through the index and then verifies that the boundary object indeed matches the boundary of the polygon. It then constructs references differently, depending on the level of the identified AdministrativeUnit. As a result of this script, an AdministrativeBoundary target object can have references to up to six AdministrativeUnits - if it happens to be a boundary that separates 5th level, 4th level and 2nd level on both sides.

The execution context is in a Groovy Retype function.

This is the full script:

This is the second installment of our “Groovy week”. This time, we are going to look into a powerful feature in hale studio that is used by the Spatial Join function, but can also be accessed through scripting - the spatial index.

Like the previous post, note that this article assumes you have working knowledge of hale studio and know the terminology.

Tuesday’s Script: Build Polygons from Lines (Florian)

We used this script to build up references from boundaries to four levels of administrative units. In the original data, such references were not explicit, with the exception of international borders.

The script first identifies possible candidate AdministrativeUnit polygon geometries through the index and then verifies that the boundary object indeed matches the boundary of the polygon. It then constructs references differently, depending on the level of the identified AdministrativeUnit. As a result of this script, an AdministrativeBoundary target object can have references to up to six AdministrativeUnits - if it happens to be a boundary that separates 5th level, 4th level and 2nd level on both sides.

The execution context is in a Groovy Retype function.

This is the full script:



You can download this script here and import it in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that it uses some protected functions, so you need to “Lift Groovy Restrictions” to execute the script. It assumes a certain geometry attribute name (the_geom) that you might have to change. The additional attributes accessed here, such as BFS_NUMMER, BEZIRKSNUM, KANTONSNUM and ICC, will not be available in your source data.

We have thus included a simpler call to the spatial index as a second example:



Happy transforming!

(more)

News' image preview

When you try to do something in hale studio that isn’t possible with the out-of-the-box functionality, hale offers several ways of creating that functionality yourself. The most accessible of these is to add your own transformation functions using Groovy scripting. There are several points where Groovy functions can be added:

  • Type Transformation Functions
  • Property Transformation Functions
  • Custom Functions
  • Post-processing scripts (only through the CLI)

Groovy is superset of Java, i.e. any valid Java program is also a valid Groovy program. What it makes much easier than Java is the creation of data structures – no boilerplate constructors and initialisation and the like. Here are some key resources for learning Groovy:

Often, you will be looking for some recipes and snippets to get started, and that is exactly what we will provide during this week: We will publish one Groovy Snippet per day. Each of my colleagues has selected their favourite script and will share it with all of you 😊.

Please note that this article assumes you have working knowledge of hale studio and know the terminology.

When you try to do something in hale studio that isn’t possible with the out-of-the-box functionality, hale offers several ways of creating that functionality yourself. The most accessible of these is to add your own transformation functions using Groovy scripting. There are several points where Groovy functions can be added:

  • Type Transformation Functions
  • Property Transformation Functions
  • Custom Functions
  • Post-processing scripts (only through the CLI)

Groovy is superset of Java, i.e. any valid Java program is also a valid Groovy program. What it makes much easier than Java is the creation of data structures – no boilerplate constructors and initialisation and the like. Here are some key resources for learning Groovy:

Often, you will be looking for some recipes and snippets to get started, and that is exactly what we will provide during this week: We will publish one Groovy Snippet per day. Each of my colleagues has selected their favourite script and will share it with all of you 😊.

Please note that this article assumes you have working knowledge of hale studio and know the terminology.

The Groovy Logo, by Zorak1103, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=13358930

Monday’s Script: Build Polygons from Lines (Thorsten)

This script is my personal favorite. While relatively complex, it shows a lot of useful approaches how to work with geometries in a Groovy script.

This function was built for creating waterbody Polygons from LineString shore segments. It is executed in the context of a Merge type cell as a Greedy Groovy Script function. It takes a MultiLineString geometry with unsorted individual LineStrings and builds one to many polygons in a MultiPolygon from that input.

This is the script:



You can download this script here and import it in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that it uses some protected functions, so you need to “Lift Groovy Restrictions” to execute the script. It furthermore assumes a certain geometry attribute name (the_geom) that you might have to change.

Happy transforming!

(more)