MMD

The MMD format is a new storage type developed as the successor to the FSS storage. It uses memory-mapped files, which makes it significantly faster than any other storage currently available. An MMD file includes metadata about the value type of each attribute and can optionally auto-declare these attributes in your configuration when reading the file.

Compared to fss, the MMD format consolidates data into one or two files per attribute, instead of producing a separate file for each tile. This avoids the creation of thousands of small files for large datasets and improves both file-system efficiency and loading performance.

Writing an MMD file can work the same way as writing a shapefile: you export a unit to disk. Reading an MMD file follows the same pattern and integrates seamlessly into existing workflows.

unit<uint64> store_mmd := Result
,	StorageName = "somestoragename.mmd"
{
	attribute<string>                                   OrgName                     := Result/OrgName;
	attribute<string>                                   DestName                    := Result/DestName;
	attribute<s>                                        Traveltime                  := Result/Traveltime;
	attribute<ct>                                       Price                       := Result/Price;
	attribute<ct>                                       Price_augm                  := Result/Price_augm;
	attribute<string>                                   ModeUsed                    := Result/ModeUsed;
	attribute<dam>                                      TravelDist_Bus              := Result/TravelDist_Bus;
	attribute<dam>                                      Traveldist_Metro            := Result/Traveldist_Metro;
	attribute<dam>                                      Traveldist_Tram             := Result/Traveldist_Tram;
	attribute<dam>                                      Traveldist_Rail             := Result/Traveldist_Rail;
	attribute<dam>                                      Traveldist_Ferry            := Result/Traveldist_Ferry;
	attribute<s>                                        W_time                      := Result/W_time, Descr = "Waiting at home";
	attribute<s>                                        V_time                      := Result/V_time;
	attribute<s>                                        PT_time                     := Result/PT_time;
	attribute<s>                                        N_time                      := Result/N_time;
}

unit<uint64> read_mmd
: StorageName = "somestoragename.mmd"
, StorageReadOnly = "true"
{

}

However, an MMD storage can also store multiple tables and attributes with different domains. Domains can be stored in the MMD, as shown in the example above, or be defined elsewhere and reused when reading the MMD data. When domains are stored in the MMD and then read back, they are unrelated to the original domain. When defined elsewhere, the user is responsible for not changing the domain between producing the MMD and using it.

Furthermore, the user is responsible for first producing MMD data before using it.

the dictionary

Next to the data files, an MMD folder holds a 0Dictionary.dms describing what was written: the value type of each attribute and the Range of each Unit that is stored in the MMD. A domain that is stored in the MMD can only be read back when its range is recorded there, so this entry is what makes such a domain usable.

For a two-dimensional unit the range is written with tagged coordinates since GeoDMS 20.14.0:

unit<ipoint> sub2d:
	Range = "[xy(0; 0), xy(5; 4)) "

Dictionaries written by older versions use the untagged {row, col} form and keep being read correctly; see XY order. Note that a dictionary written by 20.14.0 or later is not readable by an older GeoDMS if it contains a two-dimensional unit.

what the dictionary records about units defined elsewhere

A stored attribute may refer to a unit that is not stored in the MMD — a values unit such as meter, or a domain that the configuration declares itself. Such a unit is referenced by name, and that name is resolved again, in the reading configuration, against whatever is declared there under that name. When that declaration changed since the data was written, the reader used to bind the new element type to the old bytes silently: the case that produced garbage, or a crash, when an fpoint values unit had become dpoint.

Since GeoDMS 20.14.0 the dictionary therefore records what the bytes were written against, as an IntegrityCheck on the dictionary root:

container store:
	IntegrityCheck = "PropValue(dom, 'ValueType') == 'uint32' && LowerBound(dom) == uint32(0) && UpperBound(dom) == uint32(50) && PropValue(vu, 'ValueType') == 'float32'"

• the value type is recorded for every external unit;
• for an external domain the extent is recorded as well, when it is known at the moment the dictionary is written.

Units stored inside the MMD describe themselves and get no entry. Because the restriction sits on the root of the dictionary, it guards every attribute read through it, so a reading configuration that has changed one of these units fails loudly instead of reinterpreting the stored bytes.

how a reader is expected to declare the storage

A reading configuration names the storage and nothing else: the sub-items come from the dictionary. Two shapes are refused rather than half-honoured:

• sub-items declared by the reader under a read-only MMD holder — the dictionary defines those, and a second declaration can disagree with it;
• an IntegrityCheck on the read holder, which would collide with the restrictions the dictionary puts there.

An IntegrityCheck is refused on the writing side too, on any item below a storage holder that is being written: it would suppress the dictionary altogether, leaving an .mmd that cannot be read back. Configure the check on the storage holder instead.

since

The format was available as early as 2024, but its initial problems have been resolved in version 18.1.2.