News > groovy

In our fifth and final installment of the 2020 Groovy Week, we will address a type of dataset that is often complex and has specific issues that need to be solved - a transport network, for which we want to create topologically correct link sequences. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Most of the time, the order in which properties are written to a feature doesn’t really matter. However, in some cases the correct order is very important, e.g. for link sequence features that are common in INSPIRE transport Network data sets.

In this post, we will thus examine how we can use sorting in a Groovy Script to ensure that we get the correct order of references to a set of RoadLink features. In this data set, there was no single simple property that could be used to determine the correct sequence, but rather a combination of sequence and linear referencing properties that would need to be used.

The context in which this script is used is in a Groovy Script (greedy) property-level function, with a Merge type-level function. The Merge function acts like a Group By in SQL - it creates groups of features based on shared properties. In this data set, we merge on the ROUTE_ID property, a foreign key that indicates to which RoadLinkSequence a RoadLink should belong. In such a Merge case, each source property may be available many times. A greedy Groovy function will take all these instances of source properties and provide them as a list to the script. You will notice, that when accessing the source feature’s properties, we are using only lists.

To perform the actual sorting, the script builds a sortable composite key as a single string, and inserts these keys into a TreeMap. A TreeMap is sorted according to the natural ordering of its keys, so when using simple strings, we do not need to come up with our own sorting or comparator approach.

Building a working composite key was the main challenge here - we kept getting key collisions for various reasons, but in the end, the approach below produced the correct order and was free of any collisions.



Towards the end of the script, the keys and RoadLink IDs are stored in the sortedEdgeIDMap, over whose sorted value set we then iterate to emit the target DirectedLink properties in the correct order.

You can download the script snippet here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to Lift Groovy Restrictions to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

In our fifth and final installment of the 2020 Groovy Week, we will address a type of dataset that is often complex and has specific issues that need to be solved - a transport network, for which we want to create topologically correct link sequences. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Most of the time, the order in which properties are written to a feature doesn’t really matter. However, in some cases the correct order is very important, e.g. for link sequence features that are common in INSPIRE transport Network data sets.

In this post, we will thus examine how we can use sorting in a Groovy Script to ensure that we get the correct order of references to a set of RoadLink features. In this data set, there was no single simple property that could be used to determine the correct sequence, but rather a combination of sequence and linear referencing properties that would need to be used.

The context in which this script is used is in a Groovy Script (greedy) property-level function, with a Merge type-level function. The Merge function acts like a Group By in SQL - it creates groups of features based on shared properties. In this data set, we merge on the ROUTE_ID property, a foreign key that indicates to which RoadLinkSequence a RoadLink should belong. In such a Merge case, each source property may be available many times. A greedy Groovy function will take all these instances of source properties and provide them as a list to the script. You will notice, that when accessing the source feature’s properties, we are using only lists.

To perform the actual sorting, the script builds a sortable composite key as a single string, and inserts these keys into a TreeMap. A TreeMap is sorted according to the natural ordering of its keys, so when using simple strings, we do not need to come up with our own sorting or comparator approach.

Building a working composite key was the main challenge here - we kept getting key collisions for various reasons, but in the end, the approach below produced the correct order and was free of any collisions.



Towards the end of the script, the keys and RoadLink IDs are stored in the sortedEdgeIDMap, over whose sorted value set we then iterate to emit the target DirectedLink properties in the correct order.

You can download the script snippet here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to Lift Groovy Restrictions to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

(more)

In our fourth installment of the 2020 Groovy Week, we are going to take a look at how to map attributes in Groovy type transformation functions and how to use that to create multiple target objects for a single source object. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Thursday’s Script: Mapping Attributes on the Type Level to Split a Feature (Florian)

The normal approach to map values of source attributes to the target is to invoke an attribute transformation function like Rename, Classification or Groovy Script. These attribute-level functions let you create a target value based on the value(s) of the source attribute(s). Here is an example for a simple Rename that maps the source description attribute to the target attribute of the same name. The function is part of the Retype of the source type River to the target type Watercourse.

If we replace the type transformation function Retype by a Groovy Retype, we can achieve the same result as with the Rename function by using the _target instruction in the script:



A typical use case for this is if you need to split one feature into more than one target object with different target attribute values. Here is an example from an INSPIRE Hydrography alignment where one target instance is created for every source geometry:



You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

In our fourth installment of the 2020 Groovy Week, we are going to take a look at how to map attributes in Groovy type transformation functions and how to use that to create multiple target objects for a single source object. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Thursday’s Script: Mapping Attributes on the Type Level to Split a Feature (Florian)

The normal approach to map values of source attributes to the target is to invoke an attribute transformation function like Rename, Classification or Groovy Script. These attribute-level functions let you create a target value based on the value(s) of the source attribute(s). Here is an example for a simple Rename that maps the source description attribute to the target attribute of the same name. The function is part of the Retype of the source type River to the target type Watercourse.

If we replace the type transformation function Retype by a Groovy Retype, we can achieve the same result as with the Rename function by using the _target instruction in the script:



A typical use case for this is if you need to split one feature into more than one target object with different target attribute values. Here is an example from an INSPIRE Hydrography alignment where one target instance is created for every source geometry:



You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

(more)

In our third installment of the 2020 Groovy Week, we are going to cover a typical use case of hale studio’s Collector feature: building a bounding box covering the geometries of all source objects. For an introduction to the powerful Collector feature, make sure to check out our previous post on the subject. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Wednesday’s Script: Building a Bounding Box using a Collector (Florian)

A typical use case for calculating a bounding box covering the geometries of all source objects is creating an alignment to the INSPIRE Species Distribution theme. Transforming to this theme typically comprises creating several SpeciesDistributionUnit instances and a single SpeciesDistributionDataset instance which references those units.

When creating the SpeciesDistributionDataset instance, one of the mandatory target attributes to fill is domainExtent which should contain the geographic extent of all contained SpeciesDistributionUnit instances. To achieve this, we will create a bounding box covering all SpeciesDistributionUnit geometries on the fly during the transformation using a Collector. While there exists the specialized function Compute Extent for calculating bounding boxes, using it would require a Merge over all source instances. Using a Collector is more efficient in this case.

Instead of using the Rename function to map the source geometry to the SpeciesDistributionUnit, use the following Groovy script:



To assign the calculated bounding to the domainExtent property of the SpeciesDistributionDataset type, use the script below:



Make sure to reduce the priority of the SpeciesDistributionDataset transformation to make sure that it is executed last, i.e. after all SpeciesDistributionUnits are created.

You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

In our third installment of the 2020 Groovy Week, we are going to cover a typical use case of hale studio’s Collector feature: building a bounding box covering the geometries of all source objects. For an introduction to the powerful Collector feature, make sure to check out our previous post on the subject. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.

Wednesday’s Script: Building a Bounding Box using a Collector (Florian)

A typical use case for calculating a bounding box covering the geometries of all source objects is creating an alignment to the INSPIRE Species Distribution theme. Transforming to this theme typically comprises creating several SpeciesDistributionUnit instances and a single SpeciesDistributionDataset instance which references those units.

When creating the SpeciesDistributionDataset instance, one of the mandatory target attributes to fill is domainExtent which should contain the geographic extent of all contained SpeciesDistributionUnit instances. To achieve this, we will create a bounding box covering all SpeciesDistributionUnit geometries on the fly during the transformation using a Collector. While there exists the specialized function Compute Extent for calculating bounding boxes, using it would require a Merge over all source instances. Using a Collector is more efficient in this case.

Instead of using the Rename function to map the source geometry to the SpeciesDistributionUnit, use the following Groovy script:



To assign the calculated bounding to the domainExtent property of the SpeciesDistributionDataset type, use the script below:



Make sure to reduce the priority of the SpeciesDistributionDataset transformation to make sure that it is executed last, i.e. after all SpeciesDistributionUnits are created.

You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

(more)

News entry thumbnail
Groovy Week 2020: Logging and Error Handling
05.05.2020 by Florian Esser, Thorsten Reitz

In our second installment of the 2020 Groovy Week, we are going to cover the topic of adding messages to the transformation log as well as showing how to handle situations where an error needs to be raised, e.g. due to incomplete or unexpected source data. Like the previous post, note that this article assumes you have working knowledge of hale studio and know the terminology.

Tuesday’s Script: Handling and logging Errors and other relevant information (Florian)

When using a Groovy script for a specific mapping task, you often need to add checks that verify that the incoming source data can actually be transformed correctly. In cases where the transformation cannot proceed (e.g. because of missing or erroneous source data), the cause needs to be reported. To achieve that, you can add error, warning and informational messages to the transformation log as well as throw exceptions to signal a critical error.

The following script is part of an alignment that transforms road data to the INSPIRE Transport Network Roads schema. It first defines two conditions hatFahrbahnachse and hatStrassenachse based on if the source instance contains a link to a AX_Fahrbahnachse or AX_Strassenachse source instance. If neither condition is met, the transformation cannot proceed. An error message is added to the transformation log and a NoResultException is thrown to indicate that the script did not create a result in this case, which is a critical error.



Not all situations are equally critical, of course. If a warning or informational message is sufficient, use one of the two other log levels that are supported by hale studio: warning and info. To emit a warning to the transformation log, use _log.warn. For informational messages, use _log.info.

All messages emitted from a Groovy script can be accessed in hale studio via the “Instance transformation” entry of a transformation run in the “Report List”, as shown in the following picture:

You can download the script snippet here and import it in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

In our second installment of the 2020 Groovy Week, we are going to cover the topic of adding messages to the transformation log as well as showing how to handle situations where an error needs to be raised, e.g. due to incomplete or unexpected source data. Like the previous post, note that this article assumes you have working knowledge of hale studio and know the terminology.

Tuesday’s Script: Handling and logging Errors and other relevant information (Florian)

When using a Groovy script for a specific mapping task, you often need to add checks that verify that the incoming source data can actually be transformed correctly. In cases where the transformation cannot proceed (e.g. because of missing or erroneous source data), the cause needs to be reported. To achieve that, you can add error, warning and informational messages to the transformation log as well as throw exceptions to signal a critical error.

The following script is part of an alignment that transforms road data to the INSPIRE Transport Network Roads schema. It first defines two conditions hatFahrbahnachse and hatStrassenachse based on if the source instance contains a link to a AX_Fahrbahnachse or AX_Strassenachse source instance. If neither condition is met, the transformation cannot proceed. An error message is added to the transformation log and a NoResultException is thrown to indicate that the script did not create a result in this case, which is a critical error.



Not all situations are equally critical, of course. If a warning or informational message is sufficient, use one of the two other log levels that are supported by hale studio: warning and info. To emit a warning to the transformation log, use _log.warn. For informational messages, use _log.info.

All messages emitted from a Groovy script can be accessed in hale studio via the “Instance transformation” entry of a transformation run in the “Report List”, as shown in the following picture:

You can download the script snippet here and import it in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

(more)

News entry thumbnail
Groovy Week 2020: More Free scripts for hale studio
04.05.2020 by Florian Esser, Thorsten Reitz, Johanna Ott

Two years ago, we published a series of posts around using Groovy scripts to extend the functionality that hale studio offers. These posts have been a favorite among you, so we’ve decided to do a reload of the original “Groovy Week” this week!

Let’s start with the basics: When you try to do something in hale studio that isn’t possible with the out-of-the-box functionality, hale studio offers several ways of creating that functionality yourself. The most accessible of these is to add your own transformation functions using Groovy scripting. There are several points where Groovy functions can be added:

  • Type Transformation Functions
  • Property Transformation Functions
  • Custom Functions
  • Post-processing scripts (only through the CLI)

Groovy is a superset of Java, i.e. any valid Java program is also a valid Groovy program. What it makes much easier than Java is the creation of data structures – no boilerplate constructors and initialisation and the like. Here are some key resources for learning Groovy:

The following posts should help you with some common challenges. Please note that this article assumes you have working knowledge of hale studio and know the terminology.

Two years ago, we published a series of posts around using Groovy scripts to extend the functionality that hale studio offers. These posts have been a favorite among you, so we’ve decided to do a reload of the original “Groovy Week” this week!

Let’s start with the basics: When you try to do something in hale studio that isn’t possible with the out-of-the-box functionality, hale studio offers several ways of creating that functionality yourself. The most accessible of these is to add your own transformation functions using Groovy scripting. There are several points where Groovy functions can be added:

  • Type Transformation Functions
  • Property Transformation Functions
  • Custom Functions
  • Post-processing scripts (only through the CLI)

Groovy is a superset of Java, i.e. any valid Java program is also a valid Groovy program. What it makes much easier than Java is the creation of data structures – no boilerplate constructors and initialisation and the like. Here are some key resources for learning Groovy:

The following posts should help you with some common challenges. Please note that this article assumes you have working knowledge of hale studio and know the terminology.

The Groovy Logo, by Zorak1103, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=13358930

Monday’s Script: Create a persistent ID for your target object (Johanna)

Sometimes, you can’t just use a property of a source object to create a unique, persistent ID on the target side. The source object might not have an unique ID, or you might have to merge several source objects. If you just use Generate sequential ID or Generate unique ID, the ID will not be persistent across multiple transformation runs.

If a combination of attributes of the source objects can be found that is unique and stable over time, the values of those attributes can be used to generate an unique ID on the target side that is persistent.

To generate an ID from several source attributes, you can use the function _.crypto.sha256 which is part of the standard Groovy helper functions. The function creates a hash for a string and, as long as the input String is stable, the generated hash value will also be stable.

When using the generated hash as an ID, you should always use a prefix (e.g. the feature type name followed by an underscore) as the hash value might start with a number which is not allowed in a gml:id.

Here is an example for mapping the gml:id attribute of a Road object of in the INSPIRE Transport Network Roads schema:



You can download this script here and import it in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.

Happy transforming!

(more)