Module 3, Meta scripting ‐ templates and for_each

learning objective: learning how to use templates, indirect expressions and for_each statements to write reusable and scalable GeoDMS configurations

introduction

As GeoDMS configurations grow in size and complexity, repeating the same model logic for multiple scenarios, datasets, or time periods quickly becomes tedious and error-prone. Meta scripting addresses this by providing three closely related mechanisms:

  • Templates: define reusable model logic once and instantiate it multiple times with different inputs.
  • Indirect expressions: construct expressions dynamically from other data items in the configuration, enabling truly generic model logic.
  • for_each: generate a set of tree items automatically, based on a list or the contents of a data item.

Together, these concepts bring a level of abstraction to GeoDMS scripting comparable to functions, expressions and loops in a general-purpose programming language. They are essential for writing clean, maintainable models and for scenario analyses where the same calculations need to be applied across many variants.

Reading tip: for background on tree items, expressions and namespaces, see Module 1

templates

what is a template?

In GeoDMS, a template is a container of calculation rules (model logic) that includes at least one case parameter: an input whose value is not yet known at definition time. A template can be compared to a function in a programming language: the logic is defined once, and you call it with different arguments to produce different results.

The formal definitions:

A template is defined as a set of calculation rules (model logic) with at least one case parameter.

A case is defined as an instantiated template with values set for each case parameter.

how to define a template

A template is defined using the template keyword followed by the template name and a set of subitems enclosed in curly braces:

template MyTemplate_T
{
  // begin case parameters
  attribute<float32> input_attribute (domain);
  // end case parameters

  attribute<float32> result (domain) := input_attribute * 2.0;
}

Key points:

  • The case parameters are configured as the first subitems of the template, before any model logic.
  • It is good practice to mark the beginning and end of the case parameters with comments, as shown above.
  • Case parameters are not explicitly marked in the syntax; the convention of using comments makes the configuration much easier to read and maintain.
  • Do not use existing GeoDMS function names as template names.

Reading tip: for more information on case parameters, see the page: case parameter

default values for case parameters

A case parameter can optionally be given a default value via an expression or primary data:

template MyTemplate_T
{
  // begin case parameters
  attribute<float32> input_attribute (domain) := const(0.0, domain);
  // end case parameters

  attribute<float32> result (domain) := input_attribute * 2.0;
}

If a default is set, the template can be instantiated without specifying a value for that parameter, and the default will be used. Values supplied at instantiation always override the defaults.

case instantiation

To use a template, you create a case instantiation: a container whose expression calls the template name and provides values for each case parameter.

container Result_Scenario_A := MyTemplate_T('Scenario_A');
container Result_Scenario_B := MyTemplate_T('Scenario_B');
container Result_Scenario_C := MyTemplate_T('Scenario_C');

Each instantiation creates a separate subtree with all the items defined in the template, but calculated using the supplied case parameter values. This makes it straightforward to compare results across scenarios in the GeoDMS GUI.

Reading tip: for more information, see the page: case instantiation

a practical template example

The following example defines a template that calculates accessibility scores for a given set of facilities, then aggregates the results to multiple spatial levels:

template CalcAccessibility_T
{
  // begin case parameters
  attribute<meter>  dist_to_facility (residential/vbo);
  parameter<string> facility_label;
  // end case parameters

  attribute<meter>  dist_clipped (residential/vbo) := min_elem(dist_to_facility, 5000[meter]);

  container by_neighbourhood := AggregateToNeighbourhood_T(dist_clipped);
  container by_municipality  := AggregateToMunicipality_T(dist_clipped);
}

container Accessibility
{
  container supermarket    := CalcAccessibility_T(distances/supermarket,    'Supermarket');
  container general_pract  := CalcAccessibility_T(distances/general_pract,  'GP');
  container primary_school := CalcAccessibility_T(distances/primary_school, 'Primary school');
}

Because all three cases use the same template, a change to the model logic (e.g. changing the clipping distance) only needs to be made in one place.

nesting templates

Templates can call other templates. This allows you to build up a hierarchy of reusable components:

template AggregateToNeighbourhood_T
{
  // begin case parameters
  attribute<meter> dist_values (residential/vbo);
  // end case parameters

  attribute<meter> mean_dist (neighbourhoods) := mean(dist_values, residential/neighbourhood_rel);
}

The template CalcAccessibility_T above calls AggregateToNeighbourhood_T and AggregateToMunicipality_T, which are themselves templates. This nested structure is a common pattern in larger GeoDMS models.

indirect expressions

what is an indirect expression?

A regular GeoDMS expression refers directly to named items in the configuration tree: population / area. An indirect expression takes this one step further: it constructs an expression as a string at configuration time, using values from other items, and then evaluates the resulting string as a calculation rule.

An indirect expression is defined as an expression in which one or more elements are based on the contents of other items in the configuration. The resulting direct expressions are generated using the indirect expression logic and the items used in the configuration.

This is the mechanism that makes truly generic model logic possible. Where a template replaces concrete items with case parameters, an indirect expression can dynamically build the name of an item to reference, the structure of a calculation, or both.

syntax

An indirect expression always starts with a = character (an equals sign, placed immediately before the opening single quote):

attribute<bool> total (domain) := ='SomePath/'+ScenarioName+'/SomeAttribute';

Or create a list within, for example, the or-operator:

attribute<bool> total (domain) := ='or(' + asItemList(LandUseTypes/Name) + ')';

Within the indirect expression, two kinds of elements are mixed:

  • Fixed parts are written between single quotes. They appear verbatim in the resulting direct expression. In the example above, 'or(' and ')' are fixed parts.
  • Generated parts are GeoDMS expressions that must evaluate to a string. Their result is inserted into the expression string. In the example, asItemList(LandUseTypes/Name) evaluates to something like 'residential, commercial, industrial', which is spliced into the expression.

two-step processing

Indirect expressions are processed in two steps:

  1. The indirect expression is evaluated to a direct expression (a plain string). In the example above, the result might be: or(residential, commercial, industrial).
  2. The direct expression is calculated as a normal GeoDMS expression.

You can inspect the resulting direct expression in the GeoDMS GUI: open the Detail Page for the item and look under General > CalculationRule. This is the first thing to check when debugging an indirect expression.

a simple example

The following configuration calculates whether a grid cell belongs to any of a dynamic list of land-use categories:

unit<uint32> LandUseTypes : nrofrows = 3
{
  attribute<string> Name : ['residential', 'commercial', 'industrial'];
}

attribute<bool> is_any_type (grid) := ='or(' + asItemList(LandUseTypes/Name) + ')';

The helper function asItemList joins the values of an attribute into a comma-separated string. The resulting direct expression evaluated by GeoDMS is:

or(residential, commercial, industrial)

Adding a new land-use type to the LandUseTypes unit automatically includes it in the calculation — without touching the expression itself.

indirect expressions in for_each

Indirect expressions are closely tied to for_each (see the next section). When for_each generates a set of items using the e (expression) option, the expression argument is a string attribute: each element of the domain unit gets its own expression, constructed from that string.

The conventional pattern uses [rel] as an index placeholder, referring to the relation of the current element back to its domain:

container RegionGrids := for_each_nedv(
    Regions/Name                                        // [n]ame
    ,'SourceData/' + Regions/Name + '[grid/region_rel]' // [e]xpression per item (indirect)
    ,grid                                               // [d]omain unit
    ,float32                                            // [v]alues unit
);

For the region named 'Utrecht', the generated direct expression is:

SourceData/Utrecht[grid/region_rel]

This selects the Utrecht column from source data and maps it to the grid domain via grid/region_rel. Each generated item gets its own expression, tailored to its own name.

indirect expressions in conditions

Indirect expressions can be combined with the iif function to build conditional calculations that only evaluate the relevant branch:

attribute<float32> result (domain) :=
    = use_scenario_A
      ? 'ScenarioA/values'
      : 'ScenarioB/values';

In a normal (direct) iif, both the then- and else-branches are fully calculated. With an indirect expression, only the selected branch ends up in the resulting direct expression — the other is never calculated at all. This is useful for performance and for avoiding errors when one branch refers to items that may not exist in all configurations.

be aware: debugging

Indirect expressions make configurations more powerful but also harder to debug. When an error occurs in a direct expression that was generated indirectly, the error message refers to the generated expression, not to the indirect expression as written in your .dms file. This makes it harder to locate the problem in your script.

A common mistake is forgetting to wrap a fixed part in single quotes:

// wrong: const(0[woning], AllocRegio) is interpreted as a generated part, causing a type error
attribute<woning> result (AllocRegio) := = condition ? 'OtherItem' : const(0[woning], AllocRegio);

// correct:
attribute<woning> result (AllocRegio) := = condition ? 'OtherItem' : 'const(0[woning], AllocRegio)';

When debugging, always check General > CalculationRule in the Detail Page first — it shows you the direct expression that was generated and evaluated.

Reading tip: for more information, see the page: error tracking in indirect expressions; for the general debugging workflow (reading errors, FailReason, testing intermediate items), see Module 1e, Learning the basic concepts of GeoDMS, reading errors and debugging

be aware: performance

Like for_each, indirect expressions are evaluated when the GUI generates meta/scheme information, i.e. when expanding items in the tree view. If the generated or fixed parts depend on reading large primary data files or performing complex calculations, this can make the tree view slow to expand. Keep the items used inside indirect expressions as lightweight as possible.

for_each

what is for_each?

Where templates allow you to instantiate the same logic for a manually specified set of cases, for_each goes one step further: it generates a set of tree items automatically, based on the values occurring in a data item of a domain unit.

The for_each group represents a family of functions used to generate a set of items for each element of a domain unit.

This is the GeoDMS equivalent of a loop: instead of writing out container year_2000 := ..., container year_2010 := ..., container year_2020 := ... by hand, you define the logic once and let for_each create the containers for each element in a years domain unit.

for_each_ind: the modern variant

Since GeoDMS version 7.163, the original for_each family of functions is gradually being replaced by the more flexible for_each_ind function. The suffix _ind stands for indirect: the function’s options are passed as a string in the first argument, rather than being encoded in the function name itself.

It is recommended to use for_each_ind in new configurations. The older for_each_* variants are still supported for backwards compatibility. The most used versions (for_each_ne and for_each_nedv) will probably stay.

syntax

for_each:

container result := for_each_<options>(name_attribute, ...other arguments...);

for_each_ind:

container result := for_each_ind('<options>', name_attribute, ...other arguments...);

The options string controls which arguments the function expects. The possible option characters are:

Character Meaning
n name for each new item (always required, and always first)
e expression for each new item (an indirect expression string)
t template to be instantiated for each new item
d[n] domain unit; optional n if units come from a named container
v[n] values unit; optional n if units come from a named container
x value type for the generated items: creates units instead of attributes
l label property
d description property
a[t] storagename; optional t for storage type
s sqlstring property
c cdf property
u url property

The | in the notation [e|t] means exactly one of the two must be selected: either an expression (e) or a template (t), but not both.

example 1: simple parameters from a domain

Generate a parameter for each province, with the number of inhabitants as its value:

container regions := for_each_ind(
     'nedvld'                                  // options: name, expression, domain, values, label, description
    ,Relational/Region/Name                    // name attribute
    ,'Relational/Region/NrInhabitants[rel]'    // expression per item (indirect)
    ,void                                      // domain unit
    ,uint32                                    // values unit
    ,Relational/Region/Label                   // label
    ,Relational/Region/Descr                   // description
);

Given this input domain:

Name NrInhabitants Label Descr
NoordHolland 550    
ZuidHolland 1025    
Utrecht 300    
NoordBrabant 300    
Gelderland 0    

The result is a container regions with a parameter for each province, named after the province and containing the number of inhabitants. The expression 'Relational/Region/NrInhabitants[rel]' is evaluated as an indirect expression in the context of each generated item; [rel] selects the correct value per element.

example 2: reading files dynamically

Generate an attribute for each factor by reading a .tif file per factor:

container FactorData := for_each_ind(
     'ndvnda'                                          // options: name, domain, values (named), description, storagename
    ,MetaData/Factors/Name                             // name attribute
    ,Geography/Albers1kmGrid                           // domain unit
    ,Units                                             // container where values units are found
    ,MetaData/Factors/ValuesUnit                       // values unit (looked up in Units container)
    ,MetaData/Factors/Descr                            // description
    ,'%sourceDataProjDir%/' + MetaData/Factors/FileName + '.tif' // storage name per item
);

This is a common pattern when working with a large collection of raster files that share the same spatial domain but have different value types. The metadata (names, units, file paths) is stored in a small lookup table, and for_each_ind generates the reading configuration automatically.

example 3: using a template with for_each

When the generated items need subitems of their own, a template is used instead of an expression.

container PopulationByYear := 
   for_each_ne(
       period/Name                            // name attribute (e.g. '2000', '2010', '2020', '2030')
       ,'PerYear_T('+quote(period/Name)+')' // template to instantiate with the period name
   );

template PerYear_T
{
  // begin case parameters
  parameter<string> year_name;
  // end case parameters

  attribute<uint32> population (gridDomain) 
  : StorageName ="='%SourceDataDir%/pop/'+year_name+'.tif'"
  , StorageType = "gdal.grid"
  , StorageReadOnly = "TRUE";

  attribute<uint32> pop_per_municipality (municipalities) := sum(population, gridDomain/municipality_rel);
}

The result is a container PopulationByYear with a subcontainer for each year, each containing a population grid and a pop_per_municipality attribute.

example 4: reading shapefiles per category 

container Read_Shapefiles := for_each_ind(
     'nxat'
    , Services/name
    , uint32
    , '%ProjDir%/Data/src/' + Services/name + '_selection.shp'
    , 'gdal.vect'
);

Here x makes for_each_ind generate units of the given value type (here uint32) instead of attributes, and at sets both the storage name and storage type.

be aware: performance

The evaluation of for_each is triggered when the GeoDMS GUI generates the meta/scheme information for the tree — i.e. when you expand items in the tree view. If the name argument or any other argument depends on reading large primary data files or performing complex calculations, expanding items can become slow.

Advice: keep the arguments to for_each as lightweight as possible. Use small metadata tables (e.g. a simple list of names stored as a parameter or in a small FSS file) rather than deriving names from large datasets.

asitemname

The attribute used as the name argument must contain values that are valid as GeoDMS item names (alphanumeric and underscores only, not starting with a digit). If your names contain spaces or other special characters, use the asitemname function to convert them:

container results := 
    for_each_ne(
        asitemname(MyDomain/RawNames)
        ,'SomeTemplate_T('+quote(MyDomain/RawNames)+')'
    );

combining templates, indirect expressions and for_each_ind

These three mechanisms are most powerful when used together. A common pattern in real-world GeoDMS models is:

  1. Define a template that captures the full analysis for one scenario or dataset.
  2. Use indirect expressions inside the template to build dynamic references to source data from the case parameters.
  3. Use for_each to instantiate the template automatically for each element of a scenario domain unit.
template ScenarioAnalysis_T
{
  // begin case parameters
  parameter<string> scenario_name;
  // end case parameters

  attribute<float32> suitability (gridDomain) 
  : StorageName ="='%ProjDir%/suitability/'+scenario_name +'.tif'"
  , StorageType = "gdal.grid"
  , StorageReadOnly = "TRUE";

  attribute<float32> mean_suitability_per_region (regions) :=
      mean(suitability, gridDomain/region_rel);
}

container Scenarios := for_each_ne(
    ScenarioList/name
    ,'ScenarioAnalysis_T('+quote(ScenarioList/name)+')' 
);

Adding a new scenario requires only adding a row to ScenarioList — no changes to the analysis logic.

A further level of flexibility is possible by using an indirect expression in the template body to select between alternative calculation rules based on a case parameter:

template ScenarioAnalysis_T
{
  // begin case parameters
  parameter<string> scenario_name;
  parameter<string> scenario_type;  // e.g. 'baseline' or 'projection'
  // end case parameters

  attribute<float32> suitability (gridDomain) 
  : StorageName ="=scenario_type == 'baseline' 
                    ? '%ProjDir%/baseline/'+scenario_name +'.tif'
                    : '%ProjDir%/projections/'+scenario_name +'.tif'"
  , StorageType = "gdal.grid"
  , StorageReadOnly = "TRUE";

  attribute<float32> mean_suitability_per_region (regions) :=
      mean(suitability, gridDomain/region_rel);
}

Note that the indirect expression lives inside the double-quoted property value; the single quotes inside it delimit the fixed string parts, such as 'baseline' and the path fragments.

try it yourself!

In this exercise, you will apply templates, indirect expressions and for_each to refactor a configuration that repeats the same analysis logic for multiple land-use types.

  • Download the project here and unzip to a folder like C:/prj/GeoDMSAcademy.
  • Open the file main.dms in the GeoDMS_Academy\meta_scripting\cfg subfolder with the GeoDMS GUI.
  • The configuration currently contains three separate containers — residential, nature and agriculture — each with an identical set of calculations applied to different input data. The code works correctly, but is highly repetitive.

We ask you to refactor the configuration in three steps:

  1. Define a template that captures the shared calculation logic. Which items are the case parameters?
  2. Replace the three containers with three case instantiations of your new template.
  3. (Advanced) Add a classifications table LandUseTypes with the names of each land-use type, use an indirect expression inside the template to build the source data path dynamically from a case parameter, and use for_each to generate the three cases automatically.

Check your result against the result.dms file in the same subfolder.

Go to previous module: Module 2, Loading and storing data sources

Go to next module: Module 4, Basic analyses with vector data