Free hale»studioTraining on May 20th at 13:00 CEST (English)
Last month over a hundred users attended our introductory webinar to hale»studio. Now we offer a follow-up training – again, free of charge! The webinar on May 20th addresses hale»studio users with a beginner or intermediate level of expertise. We will therefore address aspects of data transformation that can be challenging for this user group.
To allow you an even better learning experience, you can look at the webinar topics in advance here. Just follow the links (more …) to the descriptions in the online help:
- Installing plug-ins for loading a FileGeodatabase
- Loading multiple WFS services that require a combined schema by creating a combined schema
- Loading multiple shapefiles and CSV files
- Using Join or Spatial join functions for multiple source datasets
- Using the Merge function
- Using the classification function with an Excel Mapping table
- Filtering attributes by setting condition contexts on source data attributes
- Mapping filtered data by using the Assign Bound function
- Visualizing data by filtering them in the views for source data and for transformed data and displaying them in the map view
- Selecting instances in the map view and displaying them in the views for source data and for transformed data
You can register for the webinar here…
…or register for the webinar by sending an email with your contact details to info@wetransform.to and enter Registration for TOP 10 Challenges as subject.
Happy transforming!
Free hale»studioTraining on May 20th at 13:00 CEST (English)
Last month over a hundred users attended our introductory webinar to hale»studio. Now we offer a follow-up training – again, free of charge! The webinar on May 20th addresses hale»studio users with a beginner or intermediate level of expertise. We will therefore address aspects of data transformation that can be challenging for this user group.
To allow you an even better learning experience, you can look at the webinar topics in advance here. Just follow the links (more …) to the descriptions in the online help:
- Installing plug-ins for loading a FileGeodatabase
- Loading multiple WFS services that require a combined schema by creating a combined schema
- Loading multiple shapefiles and CSV files
- Using Join or Spatial join functions for multiple source datasets
- Using the Merge function
- Using the classification function with an Excel Mapping table
- Filtering attributes by setting condition contexts on source data attributes
- Mapping filtered data by using the Assign Bound function
- Visualizing data by filtering them in the views for source data and for transformed data and displaying them in the map view
- Selecting instances in the map view and displaying them in the views for source data and for transformed data
You can register for the webinar here…
…or register for the webinar by sending an email with your contact details to info@wetransform.to and enter Registration for TOP 10 Challenges as subject.
Happy transforming!
(more)
After last week’s focus on Groovy, today we will take a closer look on other advanced functions and functionalities of hale studio that often get overlooked but provide great value. Making good use of these functionalities can help you on your way to becoming a more efficient hale studio power user.
For the remainder of this article it’s assumed that you have working knowledge of hale studio and are familiar with its terminology. For a primer on hale studio concepts, please refer to the online help.
Synchronize Schema Explorer and Alignment View
In large and complex source and/or target schemas, it can become difficult to find a specific source type or attribute. If you have the type or attribute relation open in the Alignment view, the button “Synchronise with cell selection” (highlighted in the screenshot below) can simplify navigation. When active, hale studio will jump to the source and target types or attributes of the selected mapping in the Schema Explorer.
Spatial Join
An advanced feature of hale studio is its ability to join objects of your source dataset based on the spatial relationship of their geometries with the Spatial Join transformation function.
In contrast, the regular Join is limited to the equality of textual attributes.
For the comparison of geometries, Spatial Join supports all modes of the Dimensionally Extended nine-Intersection Model: contains, covered by, covers, crosses, equals, intersects, overlaps, touches, and within.
Internally, Spatial Join is powered by a spatial index that is built when hale studio imports the source data.
You can leverage the spatial index in your own Groovy scripts via the helper functions _.geom.spatialIndexQuery and _.geom.boundaryCovers.
Make sure to check out our previous post on the subject if you would like to learn more about using the spatial index in Groovy scripts!
Filtering Data with Contexts
In hale studio, a context on a schema element can be used to restrict the element to certain instances or property values. After setting a context on a schema element, the effect of mapping cells defined on that element is limited to the instances or property values that match the context condition.
A special type of context is the index context that can be used in cases where the order of elements in the source instance conveys a special meaning. In the example below, the information on the language of a name is given implicitly through the order in which the names appear. The first name is in German, the second name is in English, and the third one is in Italian.
More information about contexts is available in the hale studio help. There you can also find a tutorial on how to create contexts.
Make the Most of the Map View
The Map view of hale studio provides you with a cartographic representation of both source and transformed data.
Both can be displayed alongside each other, with different layouts to choose from.
An often overlooked feature of the Map view is that you can select geometries by clicking on them and then inspect the properties of the corresponding object in the data view.
You can also select multiple instances, just hold the Ctrl key when you click.
Selected instances are highlighted in red, as can be seen in the screenshot below.
You can find the data views for source and transformed instances directly beneath the map.
To show the properties of the objects selected in the map, activate the Use instances selected in the application mode in the upper left corner of the data view.
Vice versa you can also filter the data view and highlight the geometries of the filtered instances in the map.
To do that, first activate the Random/filtered source data instances mode in the data view (upper left corner), then enter the desired filter expression and finally activate the globe button in the upper right corner of the data view.
In the screenshot below, the source objects were filtered for a specific value of the bodenart property.
The geometries of the filtered objects are highlighted in red in the map.
Happy transforming!
After last week’s focus on Groovy, today we will take a closer look on other advanced functions and functionalities of hale studio that often get overlooked but provide great value. Making good use of these functionalities can help you on your way to becoming a more efficient hale studio power user.
For the remainder of this article it’s assumed that you have working knowledge of hale studio and are familiar with its terminology. For a primer on hale studio concepts, please refer to the online help.
Synchronize Schema Explorer and Alignment View
In large and complex source and/or target schemas, it can become difficult to find a specific source type or attribute. If you have the type or attribute relation open in the Alignment view, the button “Synchronise with cell selection” (highlighted in the screenshot below) can simplify navigation. When active, hale studio will jump to the source and target types or attributes of the selected mapping in the Schema Explorer.
Spatial Join
An advanced feature of hale studio is its ability to join objects of your source dataset based on the spatial relationship of their geometries with the Spatial Join transformation function.
In contrast, the regular Join is limited to the equality of textual attributes.
For the comparison of geometries, Spatial Join supports all modes of the Dimensionally Extended nine-Intersection Model: contains, covered by, covers, crosses, equals, intersects, overlaps, touches, and within.
Internally, Spatial Join is powered by a spatial index that is built when hale studio imports the source data.
You can leverage the spatial index in your own Groovy scripts via the helper functions _.geom.spatialIndexQuery and _.geom.boundaryCovers.
Make sure to check out our previous post on the subject if you would like to learn more about using the spatial index in Groovy scripts!
Filtering Data with Contexts
In hale studio, a context on a schema element can be used to restrict the element to certain instances or property values. After setting a context on a schema element, the effect of mapping cells defined on that element is limited to the instances or property values that match the context condition.
A special type of context is the index context that can be used in cases where the order of elements in the source instance conveys a special meaning. In the example below, the information on the language of a name is given implicitly through the order in which the names appear. The first name is in German, the second name is in English, and the third one is in Italian.
More information about contexts is available in the hale studio help. There you can also find a tutorial on how to create contexts.
Make the Most of the Map View
The Map view of hale studio provides you with a cartographic representation of both source and transformed data.
Both can be displayed alongside each other, with different layouts to choose from.
An often overlooked feature of the Map view is that you can select geometries by clicking on them and then inspect the properties of the corresponding object in the data view.
You can also select multiple instances, just hold the Ctrl key when you click.
Selected instances are highlighted in red, as can be seen in the screenshot below.
You can find the data views for source and transformed instances directly beneath the map.
To show the properties of the objects selected in the map, activate the Use instances selected in the application mode in the upper left corner of the data view.
Vice versa you can also filter the data view and highlight the geometries of the filtered instances in the map.
To do that, first activate the Random/filtered source data instances mode in the data view (upper left corner), then enter the desired filter expression and finally activate the globe button in the upper right corner of the data view.
In the screenshot below, the source objects were filtered for a specific value of the bodenart property.
The geometries of the filtered objects are highlighted in red in the map.
Happy transforming!
(more)
In our fifth and final installment of the 2020 Groovy Week, we will address a type of dataset that is often complex and has specific issues that need to be solved - a transport network, for which we want to create topologically correct link sequences. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.
Friday’s Script: Sorting complex objects for Transport Network Link Sequences (Thorsten)
Most of the time, the order in which properties are written to a feature doesn’t really matter. However, in some cases the correct order is very important, e.g. for link sequence features that are common in INSPIRE transport Network data sets.
In this post, we will thus examine how we can use sorting in a Groovy Script to ensure that we get the correct order of references to a set of RoadLink features. In this data set, there was no single simple property that could be used to determine the correct sequence, but rather a combination of sequence and linear referencing properties that would need to be used.
The context in which this script is used is in a Groovy Script (greedy) property-level function, with a Merge type-level function. The Merge function acts like a Group By in SQL - it creates groups of features based on shared properties. In this data set, we merge on the ROUTE_ID property, a foreign key that indicates to which RoadLinkSequence a RoadLink should belong. In such a Merge case, each source property may be available many times. A greedy Groovy function will take all these instances of source properties and provide them as a list to the script. You will notice, that when accessing the source feature’s properties, we are using only lists.
To perform the actual sorting, the script builds a sortable composite key as a single string, and inserts these keys into a TreeMap. A TreeMap is sorted according to the natural ordering of its keys, so when using simple strings, we do not need to come up with our own sorting or comparator approach.
Building a working composite key was the main challenge here - we kept getting key collisions for various reasons, but in the end, the approach below produced the correct order and was free of any collisions.
Towards the end of the script, the keys and RoadLink IDs are stored in the sortedEdgeIDMap, over whose sorted value set we then iterate to emit the target DirectedLink properties in the correct order.
You can download the script snippet here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to Lift Groovy Restrictions to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.
Happy transforming!
In our fifth and final installment of the 2020 Groovy Week, we will address a type of dataset that is often complex and has specific issues that need to be solved - a transport network, for which we want to create topologically correct link sequences. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.
Friday’s Script: Sorting complex objects for Transport Network Link Sequences (Thorsten)
Most of the time, the order in which properties are written to a feature doesn’t really matter. However, in some cases the correct order is very important, e.g. for link sequence features that are common in INSPIRE transport Network data sets.
In this post, we will thus examine how we can use sorting in a Groovy Script to ensure that we get the correct order of references to a set of RoadLink features. In this data set, there was no single simple property that could be used to determine the correct sequence, but rather a combination of sequence and linear referencing properties that would need to be used.
The context in which this script is used is in a Groovy Script (greedy) property-level function, with a Merge type-level function. The Merge function acts like a Group By in SQL - it creates groups of features based on shared properties. In this data set, we merge on the ROUTE_ID property, a foreign key that indicates to which RoadLinkSequence a RoadLink should belong. In such a Merge case, each source property may be available many times. A greedy Groovy function will take all these instances of source properties and provide them as a list to the script. You will notice, that when accessing the source feature’s properties, we are using only lists.
To perform the actual sorting, the script builds a sortable composite key as a single string, and inserts these keys into a TreeMap. A TreeMap is sorted according to the natural ordering of its keys, so when using simple strings, we do not need to come up with our own sorting or comparator approach.
Building a working composite key was the main challenge here - we kept getting key collisions for various reasons, but in the end, the approach below produced the correct order and was free of any collisions.
Towards the end of the script, the keys and RoadLink IDs are stored in the sortedEdgeIDMap, over whose sorted value set we then iterate to emit the target DirectedLink properties in the correct order.
You can download the script snippet here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to Lift Groovy Restrictions to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.
Happy transforming!
(more)
In our fourth installment of the 2020 Groovy Week, we are going to take a look at how to map attributes in Groovy type transformation functions and how to use that to create multiple target objects for a single source object. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.
Thursday’s Script: Mapping Attributes on the Type Level to Split a Feature (Florian)
The normal approach to map values of source attributes to the target is to invoke an attribute transformation function like Rename, Classification or Groovy Script. These attribute-level functions let you create a target value based on the value(s) of the source attribute(s). Here is an example for a simple Rename that maps the source description attribute to the target attribute of the same name. The function is part of the Retype of the source type River to the target type Watercourse.
If we replace the type transformation function Retype by a Groovy Retype, we can achieve the same result as with the Rename function by using the _target instruction in the script:
A typical use case for this is if you need to split one feature into more than one target object with different target attribute values. Here is an example from an INSPIRE Hydrography alignment where one target instance is created for every source geometry:
You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.
Happy transforming!
In our fourth installment of the 2020 Groovy Week, we are going to take a look at how to map attributes in Groovy type transformation functions and how to use that to create multiple target objects for a single source object. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.
Thursday’s Script: Mapping Attributes on the Type Level to Split a Feature (Florian)
The normal approach to map values of source attributes to the target is to invoke an attribute transformation function like Rename, Classification or Groovy Script. These attribute-level functions let you create a target value based on the value(s) of the source attribute(s). Here is an example for a simple Rename that maps the source description attribute to the target attribute of the same name. The function is part of the Retype of the source type River to the target type Watercourse.
If we replace the type transformation function Retype by a Groovy Retype, we can achieve the same result as with the Rename function by using the _target instruction in the script:
A typical use case for this is if you need to split one feature into more than one target object with different target attribute values. Here is an example from an INSPIRE Hydrography alignment where one target instance is created for every source geometry:
You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.
Happy transforming!
(more)
In our third installment of the 2020 Groovy Week, we are going to cover a typical use case of hale studio’s Collector feature: building a bounding box covering the geometries of all source objects. For an introduction to the powerful Collector feature, make sure to check out our previous post on the subject. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.
Wednesday’s Script: Building a Bounding Box using a Collector (Florian)
A typical use case for calculating a bounding box covering the geometries of all source objects is creating an alignment to the INSPIRE Species Distribution theme. Transforming to this theme typically comprises creating several SpeciesDistributionUnit instances and a single SpeciesDistributionDataset instance which references those units.
When creating the SpeciesDistributionDataset instance, one of the mandatory target attributes to fill is domainExtent which should contain the geographic extent of all contained SpeciesDistributionUnit instances. To achieve this, we will create a bounding box covering all SpeciesDistributionUnit geometries on the fly during the transformation using a Collector. While there exists the specialized function Compute Extent for calculating bounding boxes, using it would require a Merge over all source instances. Using a Collector is more efficient in this case.
Instead of using the Rename function to map the source geometry to the SpeciesDistributionUnit, use the following Groovy script:
To assign the calculated bounding to the domainExtent property of the SpeciesDistributionDataset type, use the script below:
Make sure to reduce the priority of the SpeciesDistributionDataset transformation to make sure that it is executed last, i.e. after all SpeciesDistributionUnits are created.
You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.
Happy transforming!
In our third installment of the 2020 Groovy Week, we are going to cover a typical use case of hale studio’s Collector feature: building a bounding box covering the geometries of all source objects. For an introduction to the powerful Collector feature, make sure to check out our previous post on the subject. Like in the previous Groovy Week posts, note that this article assumes you have working knowledge of hale studio and know the terminology.
Wednesday’s Script: Building a Bounding Box using a Collector (Florian)
A typical use case for calculating a bounding box covering the geometries of all source objects is creating an alignment to the INSPIRE Species Distribution theme. Transforming to this theme typically comprises creating several SpeciesDistributionUnit instances and a single SpeciesDistributionDataset instance which references those units.
When creating the SpeciesDistributionDataset instance, one of the mandatory target attributes to fill is domainExtent which should contain the geographic extent of all contained SpeciesDistributionUnit instances. To achieve this, we will create a bounding box covering all SpeciesDistributionUnit geometries on the fly during the transformation using a Collector. While there exists the specialized function Compute Extent for calculating bounding boxes, using it would require a Merge over all source instances. Using a Collector is more efficient in this case.
Instead of using the Rename function to map the source geometry to the SpeciesDistributionUnit, use the following Groovy script:
To assign the calculated bounding to the domainExtent property of the SpeciesDistributionDataset type, use the script below:
Make sure to reduce the priority of the SpeciesDistributionDataset transformation to make sure that it is executed last, i.e. after all SpeciesDistributionUnits are created.
You can download the script snippets here and here and import them in hale studio as a Groovy snippet by going to File -> Import -> Groovy Snippet. Please note that some scripts use protected functions, so you might need to “Lift Groovy Restrictions” to execute the script. Make sure you replace the placeholder attribute names with your own attribute names.
Happy transforming!
(more)