Module 2, Loading and storing data sources

learning objective: understanding how the GeoDMS connects to data sources, and how to configure reading and writing data

introduction

In Module 1 you learned how to write expressions and calculations in a GeoDMS configuration. But a model is only as useful as the data it works with. This module covers how to connect your configuration to real data files: how to read source data into the GeoDMS, and how to write results back out to files.

The GeoDMS is designed to work with data, not to collect it. A typical workflow looks like this:

  1. Load source data from files (shapefiles, GeoTiffs, CSV files, databases).
  2. Calculate results using expressions.
  3. Store selected results to output files.

Steps 1 and 3 use the same underlying mechanism: a StorageManager.

the StorageManager

A StorageManager is a software component that handles reading from or writing to a specific file format. You connect a tree item to a file by setting two properties on it:

  • StorageName: the path to the file.
  • StorageType: which StorageManager to use (i.e. which file format).

For example, gdal.vect is the StorageManager for reading vector formats such as shapefiles and CSV files via GDAL. gdal.grid handles raster formats such as GeoTiff. gdalwrite.vect and gdalwrite.grid are their write counterparts. fss is the GeoDMS native binary format.

A third property, StorageReadOnly, tells the GeoDMS whether the file connection is for reading only. Setting it to "True" protects source files from being accidentally overwritten.

The key insight is that no data is read or written until you actually request a result. The GeoDMS is lazy: it only loads a file when an item connected to it is calculated, inspected in a table view, or shown in a map. Storing works the same way: the export only happens when you trigger the result item in the GUI (by double-clicking it, or using Update Subtree, shortcut Ctrl+T).

three submodules

This module is split into three parts:

  • Module 2A: reading different data formats (CSV, shapefiles, GeoTiff).
  • Module 2B: writing results to files.
  • Module 2C: the FSS format, the GeoDMS native binary format used as a fast intermediate storage between models.

Go to previous module: Module 1d, Learning the basic concepts of GeoDMS, classifying and visualising data

Go to next module: Module 2A, Loading different data sources

Table of contents