GeoTiff
The (Geo)Tiff format is our advised format for Grid data, as it:
- is open, widely used, binary(fast) and supports multiple compression types.
- allows geo-referencing projection information to be embedded with (in an accompanying .tfw world file in the same folder) or within the GeoTiff file.
(Geo)Tiff files can be read and written to with the GeoDMS.
Read
The GeoDMS supports two ways of reading (Geo)Tiff files:
- gdal.grid: for most (Geo)Tiff files, we advise to use gdal.grid (see next subparagraph), except for:
- uint2, uint4 and boolean data.
- palette based (Geo)Tiff files
- tiff StorageManager
Both read (Geo)Tiff files compressed with ZSTD, which GDAL writes with COMPRESS=ZSTD, such as the water depth maps of the Klimaateffectatlas.
gdal.grid
The following example shows how to read a (Geo)Tiff file with Gdal.grid.
The (GeoTiff) file contains the data of a tile of the AHN, in this case: 26az2.tiff.
example
unit<spoint> AHNTile
: StorageName = "%SourceDataDir%/AHN/26az2.tif"
, StorageType = "gdal.grid"
, DialogData = "Geografie/rdc"
{
attribute<float32> GridData;
attribute<float32> ReadData (subdomain);
}
The primary data is read from the 26az2.tif file. The Projection information can also be read from this file, or from a 26az2.tfw file in the same folder. The DialogData property needs to refer to the unit for the coordinate system (RD, LatLong etc) to inform the GeoDMS on how to relate the projection information from the GeoTiff to the coordinate system of the configuration.
The geographic grid domain AHNTile is implicitly defined by the projection information and the number of rows/columns of the GeoTiff.
The GridData Attribute has this AHNTile as Domain unit and is read for the whole grid.
The ReadData attribute reads a subset of the data for another domain unit, in the example subdomain. This geographic domain unit needs to be configured and can be used to read for example the AHN data for a building or the land use for a country from a European land use file. If only a (small) selection of a GeoTiff file is needed, using such a ReadData attribute is much faster than reading the GridData and making the selection afterwards.
Data read from a GeoTiff is automatically tiled in the GeoDMS. A grid domain that is left unbound and read from the file adopts the file’s native block or strip tiling, so even striped GeoTiffs (such as AHN, stored as 10000x1 strips) are read efficiently without retiling; see GeoDMS Default Tiling.
options
GDAL Options can be configured for reading (Geo)Tiff files with gdal.grid.
See: https://gdal.org/drivers/raster/gtiff.html#open-options for a full list of all open options.
tiff StorageManager
example
The following example shows how to read a (Geo)Tiff file with the tiff StorageManager.
The (GeoTiff) file is one tile of the AHN, in this case: 26az2.tiff.
unit<spoint> AHNTile
: StorageName = "%SourceDataDir%/AHN/26az2.tif"
, DialogData = "Geografie/rdc"
{
attribute<float32> GridData;
attribute<float32> ReadData (subdomain);
}
The configuration is quite similar to the Gdal.grid variant, only the StorageType does not have to be configured.
As GDAL does not support Sub Byte Element GeoTiff files, we advise to use the Tiff StorageManager to read boolean, uint2 and uint4 data.
An 8-bit (Geo)Tiff, such as a mask, can be read into a boolean attribute with either StorageManager; every value but 0 is true.
The tiff StorageManager cannot place a rotated grid, so it refuses a rotated georeference; read such a file with gdal.grid (see Projection).
A tile or strip that the tiff StorageManager cannot read or write is an error.
TrueColor
Both the gdal.grid and the tiff StorageManager can read TrueColor (Geo)Tiff files.
The essence of a TrueColor Tiff is that the GridData contains a color value (rgb code) for each grid cell. The GeoDMS supports TrueColor Bitmap/GeoTiff images with 24 bits per pixel (1 byte for red, green and blue values). The Value type for the grid data values in the GeoDMS needs to be: uint32.
example:
unit<spoint> referentie_tiff
: StorageName = "%projDir%/data/tif/kaart.tif"
, DialogData = "Geography/point_rd"
{
attribute<uint32> GridData: DialogType ="BrushColor";
}
The property DialogType = “BrushColor” is configured to inform the GeoDMS the grid values are interpreted as colors and can be used directly to visualise grid cells in the map view.
palette based
In this variant the GridData Attribute contains index numbers for each grid cell. These index numbers are related to colors in a palette. The palette is also stored in the GeoTiff file. Based on the number of entries in the palette (related to the number of different colors in the file), the griddata of a palette based GeoTiff is usually configured with value type uint4 (for 16 colors) or uint8 (for a maximum of 255 colors).
Palette based can at the moment only be read with the tiff StorageManager.
The tiff StorageManager also reads a palette file of 4 bits into a uint8 GridData, for instance to share a colorindex domain with 8-bit files.
example:
unit<uint4> colorindex;
{
attribute<uint32> palette := Grid/PaletteData, DialogType = "BrushColor";
}
unit<spoint> referentie_tiff
: StorageName = "%projdir%/data/tiff_pa.tif"
, DialogData = "Geography/point_rd"
{
attribute<colorindex> GridData;
attribute<units/color> PaletteData (colorindex);
}
The subitem named GridData refers to the index numbers. The subitem named PaletteData is an attribute with a domain unit that is unifiable with the values unit of the GridData attribute. The PaletteData is also read from or written to the Tiff file.
The property DialogType = “BrushColor” is configured for the palette subitem of the palette domain to inform the GeoDMS that the palette values are to be interpreted as colors when visualizing the grid cells in a map view.
Other palettes (not directly read from the Tiff file) can also be used, e.g. defined explicitly in the GeoDMS configuration.
Gdal.grid reads the PaletteData of a raster without a colour table as 0 for every entry, and reads a colour table with more entries than the domain of PaletteData up to that domain; entries that the table lacks are 0, as with the tiff StorageManager.
SqlString
The SqlString Property is in use for (Geo)Tiff files for 2 purposes:
1) Aggregations If a SqlString = “number” is configured, a relation is made (if possible) between the grid data domain and the result domain based on the configured projection. This relation is used to count the number of occurrences of the configured class number in the original Tiff file for the new desired grid cell.
2) Multiple bands The following example shows how to read multiple bands (only with gdal.grid)
unit<uint32> grid_uint8_nth_band
: StorageName = "%SourceDataDir%/input/tiff_uint32.tif"
, StorageType = "gdal.grid"
, DialogData = "units/point_rd"
, StorageReadOnly = "True"
{
attribute<uint8> Red : SqlString="BANDS=1";
attribute<uint8> Blue : SqlString="BANDS=2";
attribute<uint8> Green : SqlString="BANDS=3";
attribute<uint8> Magenta : SqlString="BANDS=4";
}
A band number may be followed by other characters, as in BANDS=(1). A band that the file does not have is an error that names the band and the number of bands in the file.
Write
The GeoDMS supports two ways of writing (Geo)Tiff files:
- gdalwrite.grid: for most (Geo)Tiff files, we advise to use gdalwrite.grid (see next subparagraph), except for:
- uint2, uint4 and boolean data.
- palette based (Geo)Tiff files
- tiff StorageManager
gdalwrite.grid
!!! We noticed some issues with writing GeoTiff files with gdalwrite.grid. We advise to use the tiff StorageManager instead of gdalwrite.grid at the moment.
The following example shows how to write a (Geo)Tiff file with gdalwrite.grid.
example
unit<spoint> Export_AHNTile := AHNTile
: StorageName = "%LocalDataProjDir%/AHN/26az2.tif"
, StorageType = "gdalwrite.grid"
{
attribute<float32> GridData := AHNTile/Export_AHNTile;
}
options
GDAL Options can be configured for writing (Geo)Tiff files with gdalwrite.grid.
See: https://gdal.org/drivers/raster/gtiff.html#creation-options for a full list of all creation options.
tiff StorageManager
The following example shows how to write a (Geo)Tiff file with the tiff StorageManager.
example
unit<spoint> AHNTile
: StorageName = "%LocalDataProjDir%/AHN/26az2.tif"
{
attribute<float32> GridData:= AHNTile/Export_AHNTile;
}
In the same way as with reading, the configuration for writing GeoTiff files is quite similar to the gdalwrite.grid variant, only the StorageType does not have to be configured.
As GDAL does not support Sub Byte Element GeoTiff files, we advise to use the Tiff StorageManager to write boolean, uint2 and uint4 data.
palette based
In this variant the GridData Attribute contains index numbers for each grid cell. These index numbers are related to colors in a palette. The palette is also stored in the GeoTiff file. Based on the number of entries in the palette (related to the number of different colors in the file), the griddata of a palette based GeoTiff is usually configured with value type uint4 (for 16 colors) or uint8 (for a maximum of 255 colors).
Palette based can at the moment only be written with the tiff StorageManager.
example:
unit<uint4> colorindex
{
attribute<units/color> palette := Grid/PaletteData, DialogType = "BrushColor";
}
unit<spoint> Read:
StorageName = "%projdir%/data/tiff_pa.tif",
DialogData = "units/point_rd",
StorageReadOnly = "True"
{
attribute<colorindex> GridData;
attribute<units/color> PaletteData (colorindex);
}
container Write: StorageName = "%localDataProjDir%/regr_results/tiff_pa.tif", ExplicitSuppliers = "Read"
{
attribute<colorindex> GridData (Grid) := Grid/GridData;
attribute<units/color> PaletteData (colorindex) := Grid/PaletteData;
}
files written by older versions
Older versions differ from the description above in these respects:
- A (Geo)Tiff that the tiff StorageManager of a version before 20.23.1 wrote can lack a tile or strip, as those versions pass over a failed write without a message. They also read a damaged tile as nodata, without a message.
- A version before 20.29.0 cannot open a (Geo)Tiff compressed with ZSTD, and reports “ZSTD compression support is not configured” and “Cannot open TIFF file due to missing codec ZSTD”. Only the GLOBIO build (
.g) of such a version, whose libtiff comes from conda, has the codec. - Before 20.24.0, the tiff StorageManager refuses an 8-bit (Geo)Tiff for a boolean attribute with “Mismatch in number of bits”, and Gdal.grid reads it but takes the lowest bit of each value, so that 2 or 200 become false, and reads the last pixels of each strip or tile as false when the width of the grid is not a multiple of 8.
- Before 20.23.1, the tiff StorageManager drops the rotation of a rotated georeference without a message.
- Versions 14.1.1 up to and including 20.23.1 refuse a 24-bit TrueColor file in the tiff StorageManager with “unsupported TIFF pixel type”.
- Before 20.24.0, the tiff StorageManager refuses a palette file of 4 bits for a uint8 GridData with “Mismatch in number of bits”; only a uint4 GridData reads such a file.
- Before 20.23.1, gdal.grid crashes on the PaletteData of a raster without a colour table, and writes past the domain of PaletteData when the colour table has more entries.
- Before 20.23.1,
BANDS=(1)fails with an exception that does not say what it read, and a band that the file does not have fails an internal check.