Dbf

Dbf files are dBase files, used as part of the ESRI Shapefile or as separate data file.

In reading and writing dbf files, the value types of the values units of the attributes should match with the data types of the attributes in the dbf file.

The following value types can be used for dbf data:

• (u)int8/16/32/64 for byte, integer and long integer
• float32/64 for single/double
• bool for boolean
• string for string

Other data/value types can not be read from (e.g. date) or written to (e.g. all Point Group types) dbf storages.

Since GeoDMS 20.23.1 the dbf StorageManager reads and writes int64 and uint64, the value types to use for ids of 16 to 18 digits. Before, it read every value of such an attribute as 0, without an error. The values are read as integers, so they stay exact above 2^53, where a float64 would round them. Where the dbf StorageManager derives the value type of a column itself, as with SyncMode, a numeric column without decimals that is wider than 10 digits, N(11,0) to N(19,0), now becomes int64 instead of int32; N(10,0), the usual width of an ESRI long integer, stays int32.

Read

The GeoDMS supports two ways of reading dbf files:

• gdal.vect: we advise to use the gdal.vect StorageManager to read dbf files as it supports segmented data, see the next subparagraph.
• dbf StorageManager

gdal.vect

The following example shows how to read a .dbf file with Gdal.vect.

example:

unit<uint32> table: StorageName = "%projDir%/data/DBF.dbf"
,  StorageType     = "gdal.vect"
,  StorageReadOnly = "True"
{
  attribute<int32>   IntegerAtt;
  attribute<float32> FloatAtt;
  attribute<bool>    BoolAtt;
  attribute<string>  StringAtt;
}

dbf storagemanager

The following example shows how to read a .dbf file with the dbf storagemanager.

example:

unit<uint32> table: StorageName = "%projDir%/data/DBF.dbf"
,  StorageReadOnly = "True"
{
   attribute<int32>   IntegerAtt;
   attribute<float32> FloatAtt;
   attribute<bool>    BoolAtt;
   attribute<string>  StringAtt;
}

Since GeoDMS 20.23.1 a numeric field that holds only blanks or * characters, the way dBase, ESRI and GDAL store a NULL, is read as Null. Before, a single such field failed the whole column with the error unexpected character in parsing ‘…’ as numeric.

Since GeoDMS 20.23.1 the dbf StorageManager reads a column a block of records at a time, where it used to seek to every single field. Reading a table of 200000 records with 10 numeric columns went from about 7 seconds to under half a second on one machine; the larger the table, the more this saves.

Since GeoDMS 20.23.1 the dbf StorageManager reads and writes a .dbf above 2 GB at the right records. Before, a record beyond 2 GB was not reached, and the read went on from wherever the file was.

Write

The GeoDMS supports two ways of writing dbf files:

• gdalwrite.vect: we advise to use the gdalwrite.vect StorageManager to write dbf files, see the next subparagraph.
• dbf StorageManager

Be aware: the names of attributes written to a .dbf file may not exceed 10 characters, as the .dbf file does not support longer fields.

We advise to write attributes to dbf file that do not have Null values, as strange results might occur. Use the MakeDefined function to convert null values to a specific missing data value for your dbf file.

gdalwrite.vect

The following example shows how to write a .dbf file with the Gdalwrite.vect.

example:

unit <uint32> pc6_export := src/pc6
,  StorageName     = "%localDataProjDir%/export_table.dbf"
,  StorageType     = "gdalwrite.vect"
,  StorageReadOnly = "false"
{
   attribute<uint32>  IntegerAtt := const(1, .);
   attribute<float32> FloatAtt   := const(1f, .);
   attribute<string>  StringAtt  := const('A', .);
   attribute<bool>    BoolAtt    := const(true, .);
}

dbf storagemanager

The following example shows how to write a .dbf file with the dbf storagemanager.

example:

unit <uint32> pc6_export := src/pc6
,  StorageName     = "%localDataProjDir%/export_table.dbf"
,  StorageReadOnly = "false"
{
   attribute<uint32>  IntegerAtt := const(1, .);
   attribute<float32> FloatAtt   := const(1f, .);
   attribute<string>  StringAtt  := const('A', .);
   attribute<bool>    BoolAtt    := const(true, .);
}

Since GeoDMS 20.23.1 a write that fails, on a full disk or a share that went away, is an error, where it left a truncated .dbf that was reported as written. The header of a .dbf stores its own size and the record size in 16 bits, so a .dbf holds at most 2046 columns and records of at most 65535 bytes, the sum of the column widths; more is refused before anything is written, where it was written with a header that no reader takes. The written copy replaces the existing file in one step, also for a path with characters outside the Windows code page in the Python module; when it cannot, the existing file stays as it was, where it used to be removed first.