MMD
The MMD format is a new storage type developed as the successor to the FSS storage. It uses memory-mapped files, which makes it significantly faster than any other storage currently available. An MMD file includes metadata about the value type of each attribute and can optionally auto-declare these attributes in your configuration when reading the file.
Compared to fss, the MMD format consolidates data into one or two files per attribute, instead of producing a separate file for each tile. This avoids the creation of thousands of small files for large datasets and improves both file-system efficiency and loading performance.
Writing an MMD file can work the same way as writing a shapefile: you export a unit to disk. Reading an MMD file follows the same pattern and integrates seamlessly into existing workflows.
unit<uint64> store_mmd := Result
, StorageName = "somestoragename.mmd"
{
attribute<string> OrgName := Result/OrgName;
attribute<string> DestName := Result/DestName;
attribute<s> Traveltime := Result/Traveltime;
attribute<ct> Price := Result/Price;
attribute<ct> Price_augm := Result/Price_augm;
attribute<string> ModeUsed := Result/ModeUsed;
attribute<dam> TravelDist_Bus := Result/TravelDist_Bus;
attribute<dam> Traveldist_Metro := Result/Traveldist_Metro;
attribute<dam> Traveldist_Tram := Result/Traveldist_Tram;
attribute<dam> Traveldist_Rail := Result/Traveldist_Rail;
attribute<dam> Traveldist_Ferry := Result/Traveldist_Ferry;
attribute<s> W_time := Result/W_time, Descr = "Waiting at home";
attribute<s> V_time := Result/V_time;
attribute<s> PT_time := Result/PT_time;
attribute<s> N_time := Result/N_time;
}
unit<uint64> read_mmd
: StorageName = "somestoragename.mmd"
, StorageReadOnly = "true"
{
}
However, an MMD storage can also store multiple tables and attributes with different domains. Domains can be stored in the MMD, as shown in the example above, or be defined elsewhere and reused when reading the MMD data. When domains are stored in the MMD and then read back, they are unrelated to the original domain. When defined elsewhere, the user is responsible for not changing the domain between producing the MMD and using it.
Furthermore, the user is responsible for first producing MMD data before using it.
the dictionary
Next to the data files, an MMD folder holds a 0Dictionary.dms describing what was written: the value type of each attribute and the Range of each Unit that is stored in the MMD. A domain that is stored in the MMD can only be read back when its range is recorded there, so this entry is what makes such a domain usable.
For a two-dimensional unit the range is written with tagged coordinates since GeoDMS 20.14.0:
unit<ipoint> sub2d:
Range = "[xy(0; 0), xy(5; 4)) "
Dictionaries written by older versions use the untagged {row, col} form and keep being read correctly; see XY order. Note that a dictionary written by 20.14.0 or later is not readable by an older GeoDMS if it contains a two-dimensional unit.
what the dictionary records about units defined elsewhere
A stored attribute may refer to a unit that is not stored in the MMD — a values unit such as meter, or a domain that the configuration declares itself. Such a unit is referenced by name, and that name is resolved again, in the reading configuration, against whatever is declared there under that name. When that declaration changed since the data was written, the reader used to bind the new element type to the old bytes silently: the case that produced garbage, or a crash, when an fpoint values unit had become dpoint.
Since GeoDMS 20.14.0 the dictionary therefore records what the bytes were written against, as an IntegrityCheck on the dictionary root:
container store:
IntegrityCheck = "LowerBound(dom) == uint32(0) && UpperBound(dom) == uint32(50) && PropValue(vu, 'ValueType') == 'float32'"
- for an external domain the extent is recorded, when it is known at the moment the dictionary is written. The bound literals carry the value type (
uint32(0)), so they pin the type as well and no separate value-type restriction is written for that unit — since GeoDMS 20.17.0; earlier versions wrote both; - the value type is recorded for every other external unit: a values unit, and a domain whose extent could not be established (an unknown range, or 64-bit integral bounds);
- every unit declared outside the dictionary is named by its absolute path, in the restrictions and in the attribute declarations alike — since GeoDMS 20.17.0. A reader merges the dictionary into a container of its own naming, so any relative spelling resolves against the reader’s tree rather than the writer’s: a
../RegioUnitcannot be resolved there at all (Unknown identifier '../RegioUnit'), the dots notation...silently lands on whatever sits at that position (LowerBound Error: … arg1 of type TreeItem), and a bare name binds to whatever the reader declares under that name. Earlier versions wrote whichever spelling the writing configuration happened to use. Units inside the dictionary keep their relative form, which is the only one that survives the merge.
Units stored inside the MMD describe themselves and get no entry. Because the restriction sits on the root of the dictionary, it guards every attribute read through it, so a reading configuration that has changed one of these units fails loudly instead of reinterpreting the stored bytes.
the length of a stored array is checked against the domain
A stored attribute is a keyless positional array: element i of the file is element i of the domain, and nothing in the file itself says which domain that was. Since GeoDMS 20.16.0 the reader compares the length of every stored array with the length its domain requires, and refuses the store when the two differ:
FileTileArray Error: stored array 'C:/.../store.mmd/val' holds 40 bytes, but the domain it is read
into requires exactly 36 bytes (9 elements in 1 tile(s)). The array was written for a different
domain; recreate the storage or restore the domain it was written with.
Before that, a file holding more elements than the domain was accepted silently: the surplus was dropped and the remaining values were handed out as if they belonged to the current domain. Every value could then be attached to the wrong element while all totals stayed plausible, so nothing downstream looked wrong. A file holding fewer elements did fail, but as a Windows error from the memory mapping (CreateFileMapping ... ErrorCode 5: Access is denied) that named neither the domain nor the sizes.
For attributes of variable-length values (strings, sequences) the check covers the index array; the accompanying .seq file has no length that follows from the domain.
This check is independent of the dictionary restrictions above and does not rely on them, so it also guards stores whose dictionary carries no restriction — every .mmd written before 20.14.0. Expect it to fire on the first run after upgrading when a domain was changed after its .mmd was written: such a store must be rewritten.
how a reader is expected to declare the storage
A reading configuration names the storage and nothing else: the sub-items come from the dictionary. The reading container may be named freely: the dictionary carries the name of the container that wrote the store, and in the usual decoupling pattern the two differ — the writing project may not even be the same project. Since GeoDMS 20.17.0 that difference is no longer reported as a warning; it is a minor trace naming both, so the event log of a normal run stays clean.
Two shapes are refused rather than half-honoured:
- sub-items declared by the reader under a read-only MMD holder — the dictionary defines those, and a second declaration can disagree with it;
- an IntegrityCheck on the read holder, which would collide with the restrictions the dictionary puts there.
An IntegrityCheck is refused on the writing side too, on any item below a storage holder that is being written: it would suppress the dictionary altogether, leaving an .mmd that cannot be read back. Configure the check on the storage holder instead.
since
The format was available as early as 2024, but its initial problems have been resolved in version 18.1.2.