Module 2, Loading and storing data sources

learning objective: understanding how the GeoDMS connects to data sources, and how to configure reading and writing data
introduction
In Module 1 you learned how to write expressions and calculations in a GeoDMS configuration. But a model is only as useful as the data it works with. This module covers how to connect your configuration to real data files: how to read source data into the GeoDMS, and how to write results back out to files.
The GeoDMS is designed to work with data, not to collect it. A typical workflow looks like this:
- Load source data from files (shapefiles, GeoTiffs, CSV files, databases).
- Calculate results using expressions.
- Store selected results to output files.
Steps 1 and 3 use the same underlying mechanism: a StorageManager.
the StorageManager
A StorageManager is a software component that handles reading from or writing to a specific file format. You connect a tree item to a file by setting two properties on it:
StorageName: the path to the file.StorageType: which StorageManager to use (i.e. which file format).
For example, gdal.vect is the StorageManager for reading vector formats such as shapefiles and CSV files via GDAL. gdal.grid handles raster formats such as GeoTiff. gdalwrite.vect and gdalwrite.grid are their write counterparts. MMD and the older fss are the GeoDMS native binary formats.
A third property, StorageReadOnly, tells the GeoDMS whether the file connection is for reading only. Setting it to "True" protects source files from being accidentally overwritten.
The key insight is that no data is read or written until you actually request a result. The GeoDMS is lazy: it only loads a file when an item connected to it is calculated, inspected in a table view, or shown in a map. Storing works the same way: the export only happens when you trigger the result item in the GUI (by double-clicking it, or using Update Subtree, shortcut Ctrl+T).
four submodules
This module is split into four parts:
- Module 2A: reading different data formats (CSV, shapefiles, GeoTiff).
- Module 2B: writing results to files.
- Module 2C: the MMD and FSS formats, the GeoDMS native binary formats used as a fast intermediate storage between models.
- Module 2D: exchanging tables with Python through parquet files.
Go to previous module: Module 1e, Learning the basic concepts of GeoDMS, reading errors and debugging
Go to next module: Module 2A, Loading different data sources