How to run the model

This page tells you how to install the model and how to run it with batch\UpdateAll.cmd. It covers what you need as a user of the model. Other pages give more detail:

How the model works is on Public transport and Private transport.

Contents

Installing components

GeoDMS software

Install GeoDMS 20.27.0, with the file GeoDms20.27.0.m-Setup-x64.exe. This is the version that UpdateAll.cmd uses. It computes a car cache in about half the time of 20.26.0, with the same result.

Which functions need which version

The model needs the Pareto functions of GeoDMS 20.24.2 or later: pareto_optimal_eps with an epsilon per criterion and with bool criteria, the Pareto option of impedance_matrix, and the time-dependent impedances (#128). The car routes of an OD pair need 20.26.0 (see Routes of an OD pair). In 20.27.0, impedance_matrix makes a sparse result in one pass instead of two (GeoDMS#1327), which halves the time of a car cache.

The GeoDMS Academy explains how to install GeoDMS and how to open a configuration.

Python

The last part of the output step runs a Python script, batch\OdTellingen.py. Install Python 3; the standard library is enough. The command python must work in a command prompt, so Python must be on the PATH.

Network model project scripts

Clone the repository with Git:

git clone https://github.com/ObjectVision/NetworkModel_PBL.git

This makes a folder NetworkModel_PBL. You can also use a Git client such as TortoiseGit, or download a ZIP. The ZIP unpacks as NetworkModel_PBL-main. Rename that folder to NetworkModel_PBL (see Before the first run).

Source data

The source data are not in the repository. PBL keeps them on its Nextcloud share: the GTFS feeds, the OSM extracts, the cycling network of the Fietsersbond, the NS timetable (IFF), the OV-fiets locations, the CBS area division and the BAG. The fare tables are part of the repository (cfg/main/SourceData/OVprijzen.dms and the folder data).

The model finds the source data through three settings in cfg/main/ConfigSettings.dms:

  • NetworkModel_Dir, by default %SourceDataDir%/NetworkModel: the source data on the Nextcloud share;
  • NetworkModel_PrivDir, by default %SourceDataDir%/NetworkModel_PBL_priv: the TomTom caches (see below);
  • BAG_DataDir, by default %SourceDataDir%: the BAG snapshots of the RuimteScanner, only for Output/PostAnalysis/UAI.

%SourceDataDir% is the source data folder in your GeoDMS settings. If your folders are different, do not change ConfigSettings.dms. Set the value in GeoDmsGui under Settings > Config Options instead. You can also set it as an environment variable; see Starting UpdateAll.cmd.

The model only reads these folders. Everything it makes goes to %LocalDataProjDir%, by default C:\LocalData\NetworkModel_PBL.

The car on the TomTom network needs four TomTom caches as source data: Roads.mmd, Junctions.mmd, Speednetworks.mmd and Speedprofiles.mmd in %NetworkModel_PrivDir%/TomTom/<edition>. For Y2025 and Y2026 the edition is yr2025_12 (1.2 GB). TomTom data are licensed to PBL: get them from PBL and do not pass them on. Without these caches, set UseTomTomNetworkForCars to FALSE. The car then uses the OSM network.

Setting up a run

Open cfg/main/ModelParameters.dms and set at least these parameters:

  • AnalysisMoment: the reference year, for example 'Y2025'. It sets the analysis day, the GTFS feed and the other source data.
  • PartOfDay: 'Ochtendspits', 'Middag' or 'Avond'.
  • Orgset and Destset: the origins and the destinations, for example 'Buurt2025'.
  • Component: what the run computes: 'WW' (walking), 'CC' (cycling), 'Car' or 'PT' (public transport). For 'PT', also set PreTransport and PostTransport.
  • OutputLabel: the name of the variant in the output folder, for example 'b2025' for Buurt2025 in the morning rush and 'b2025_middag' for the afternoon. The folder name starts with the analysis date, so the year is already in it. Keep the label the same for every year that you want to compare; see OutputLabel.

All settings are on Model parameters. If GeoDmsGui is open, reload the configuration with Alt+R after each change.

The settings in group 5 at the top of ModelParameters.dms are part of the names of the caches. After you change one of them, the model makes those caches again, which can take hours. The settings in group 4 only change the output.

Run the section sources once per reference date. Which other sections of UpdateAll.cmd a run needs, depends on its component:

Component Sections
WW, CC output
Car cars, output
public transport ('PT') legs, chains, output
MergeOutput output, after the output of the merged components

Updating all caches with batch\UpdateAll.cmd

batch\UpdateAll.cmd makes the caches and the output of a run. It runs each step in a separate GeoDmsRun, so that the memory is freed between the steps. It stops at the first step that fails. It skips the steps that the component of the run does not need.

The steps run strictly one after another: the script waits for each GeoDmsRun to end before it starts the next, and so do the portions within the chain step and the output step. One process of the model is therefore computing at any moment, and the memory a run needs is the peak of its heaviest step and not the sum of the steps. Within that one process the model does use all cores, and the chain step divides its work over the stop blocks (see Block-based parallelisation), but that is parallelism inside one GeoDmsRun. Two runs at the same time only happens when you start them yourself, which for the output step is a bad idea (see Hardware requirements).

Before the first run

Check these five points. Otherwise UpdateAll.cmd stops at once, or reads the wrong folders.

  1. GeoDMS is installed (see GeoDMS software). UpdateAll.cmd looks for exactly the version on its line GEODMS_VERSION, now 20.27.0. With another version it stops with ABORT: no GeoDmsRun.exe found. Then set the environment variable GEODMS_EXE, as in Starting UpdateAll.cmd.
  2. The folder of the model is named NetworkModel_PBL (see Network model project scripts). GeoDMS names %LocalDataProjDir% after this folder. With the folder NetworkModel_PBL-main of a ZIP, the model writes and looks for its caches in %LocalDataDir%\NetworkModel_PBL-main.
  3. The source data folders are set (see Source data). GeoDmsRun reads the same settings as GeoDmsGui, so you set them only once.
  4. The run is set up in ModelParameters.dms (see Setting up a run).
  5. Python 3 is on the PATH (see Python).

The chain step and the output step need much memory; see Hardware requirements.

Starting UpdateAll.cmd

Start UpdateAll.cmd in a command prompt: press Windows+R, type cmd and press Enter. The folder of the prompt does not matter: UpdateAll.cmd finds the model next to itself and writes its logs to batch\log.

In the prompt, first set the environment variables, if you need them. Then call UpdateAll.cmd with its full path and the name of a section. An example, with the source data in K:\SD_next and the model in K:\pd\NetworkModel_PBL:

set "GEODMS_directories_SourceDataDir=K:\SD_next"
K:\pd\NetworkModel_PBL\batch\UpdateAll.cmd sources
  • GEODMS_directories_SourceDataDir sets %SourceDataDir%. It takes the place of the folder in Settings > Config Options. With K:\SD_next, the model reads the source data from K:\SD_next\NetworkModel and the TomTom caches from K:\SD_next\NetworkModel_PBL_priv. Other folders for NetworkModel_Dir or NetworkModel_PrivDir in Config Options still count. If Config Options already has K:\SD_next as the source data folder, leave this line out.
  • With another version of GeoDMS than 20.27.0, also set GEODMS_EXE to its GeoDmsRun.exe, for example set "GEODMS_EXE=C:\Program Files\ObjectVision\GeoDms20.26.0.m\GeoDmsRun.exe".
  • Put the quotes around the whole assignment, as in the examples. Do not put them around the path only: set GEODMS_EXE="C:\..." puts the quotes in the value, and then UpdateAll.cmd cannot start GeoDmsRun.exe.
  • The variables only hold in this command prompt. Run the next sections in the same prompt, for example K:\pd\NetworkModel_PBL\batch\UpdateAll.cmd legs. In a new prompt, set them again.

Do not double-click UpdateAll.cmd in Explorer. Without a section it runs all sections, which takes more than a night. The window also closes at the end, so you lose its messages. For a long run, use a scheduled task (see Running UpdateAll as a scheduled task).

The commands:

batch\UpdateAll.cmd                          all sections: sources, cars, chains, legs, output
batch\UpdateAll.cmd all -nosources           all sections except those named with -no<section>
batch\UpdateAll.cmd sources                  the source caches
batch\UpdateAll.cmd cars [moment ...]        the car caches, of the moments the run reads or of the named moments
batch\UpdateAll.cmd chains                   the public transport chain cache
batch\UpdateAll.cmd legs                     the pre- and post-transport caches that are missing
batch\UpdateAll.cmd output                   the output per origin block and departure moment

To compute one origin-destination pair, with the route of every option, see Routes of an OD pair.

The sections

  • sources makes the OSM network and the GTFS feed into caches. Run it once per reference date.
  • legs makes the walking and cycling connections between the places and the stops. Run it once per reference date and set of origins and destinations.
  • chains makes the public transport journeys between stops, the chain cache. Run it once per reference date and part of the day. This is the heaviest step of the model, in time and in memory.
  • cars makes the car trips, one cache per departure moment. Run it once per edition of the road network and set of origins and destinations.
  • output makes the result of the component: a file per block of origins and per departure moment.

How long each section takes and how much memory it needs is in the table under Hardware requirements.

A cache with the right settings is read again in a later run, so a section that finds its caches is quick. When you change a setting that a cache depends on, run its section again: the section then makes that cache again.

When a section must run again
  • sources: after a new edition of OSM or of the GTFS feed.
  • legs: after a change in the walking or cycling network or speed, in a maximum pre- or post-transport time, or in the origins, the destinations or the stops (a new feed or reference date). Not after a change in the part of the day or the component.
  • chains: after a new GTFS feed or reference date. Also after a change in the part of the day, the fare method, MinimiseCriteria, Max_transfers, MaxPTTime, the chain epsilon, the transfer settings, the maximum pre-transport times, or the start and parking time of cycling.
  • cars: after a new edition of the road network, other origins or destinations, or a change in MaxCarTime, the car front settings, the speed cap, or the window, start time or search of the time-dependent car trip. Not after a change in the car costs: the model applies them when it reads the cache.
  • output: after any of the changes above.

Which items each section asks the model for, is on UpdateAll and cache internals.

The section output first empties the output folder of the component. It removes the files that the model wrote there before, in a run with the same OutputLabel. If you want to keep earlier output, give the new run another OutputLabel.

If a run is aborted

A long run can stop halfway: a step that fails, a crash, the machine out of memory, or a restart. How much of the work the next UpdateAll.cmd has to do again differs per section, because each section has its own unit of work and its own test for what is already there:

Section Unit of work After an abort
sources one cache the interrupted cache is made again from the start
legs a base set the base sets with a cache are kept, but see below
chains a stop block the blocks with a cache are kept, the step goes on at the first block without one
cars a departure moment the moments with a complete cache are kept, the interrupted moment starts again
output the whole component everything is computed again, nothing is kept

The chain step is the one you can interrupt without losing much. A block counts as written only when GeoDMS has written the file 0Dictionary.dms in the cache of that block, which it does after all the data of the block, so a block that was half written is made again instead of being skipped. BlocksWritten shows how far the cache is, for example 500 of 500 stop blocks have a cache.

The output step does not resume. It empties the output folder before its first portion, and a portion does not look at what is already in the folder, so an output that is aborted at block 300 of 450 starts at block 1 again and the hours it had already spent are lost. Give a long output run a machine it can keep, and start it as a scheduled task (see Running UpdateAll as a scheduled task), so that it does not depend on the window you started it from.

The pre- and post-transport caches are the one case where the next run can trip over an incomplete cache. The model writes the signature of a base set before it writes the parquet file with the connections, so after an abort in between, that base set counts as written while its cache is missing or incomplete, and the next run stops with an error when it reads it. Delete the files Legs_<name>_<hash>.* of that base set in %LocalDataProjDir%\IntermediateResults and run batch\UpdateAll.cmd legs again. What marks a unit of work as done, per section, is on UpdateAll and cache internals.

Where the results are

The output of a component goes to the folder %LocalDataProjDir%\Output\<Analysis_date>_<OutputLabel>_<component>, for example Output\20251014_b2025_W_PT_W. A component that cycles over the network of the Fietsersbond gets _fb at the end of the folder name. The folder holds:

  • PerBlock\tt_<moment>_<n>of<N>.parquet: the options per origin-destination pair, one file per block of origins and per departure moment. With OutputFormat 'csv' these are csv files.
  • od_tellingen.csv and unieke_od_tellingen.csv: the number of rows and of origin-destination pairs.
  • signature.txt: all settings of the run.

The columns of the files are described under The parquet files on Public transport. How to merge the files of the blocks into larger files is on Output files.

Logs

Each step writes two files: batch\log\UpdateAll_<step>.log, the log of GeoDMS, and batch\log\UpdateAll_<step>.txt, the text of the console. When a step fails, these two files show why. GeoDmsRun adds to an existing log, so one log can hold several runs.

If the output of a public transport component stops within minutes with a long list of Unknown identifier errors, first about columns of the chain cache such as uq_c_FromTime_Stop_rel and then about columns of the output such as OrgName, the model found no chain cache with the settings of the run. The reader of the chain cache declares no columns itself, so without a cache every item that uses it is unknown. The cause is in the warning just before the first error: the IntegrityCheck of /NetworkSetup/PublicTransport_Prep/PT_Chains_Sig/CheckGate is not true. The chain cache was then never made, was removed, or was made with other settings. Run batch\UpdateAll.cmd chains and then output again. The section output does not check beforehand whether the chain cache exists, so it has already emptied the output folder of its OutputLabel.

Running UpdateAll as a scheduled task

A run of many hours should not depend on the window that started it. batch\ScheduleUpdateAll.cmd therefore starts UpdateAll.cmd as a scheduled task of Windows, named NM_PBL_UpdateAll, and returns at once:

K:\pd\NetworkModel_PBL\batch\ScheduleUpdateAll.cmd chains
  • The task runs in a console window that you can see. The run goes on when you close the window that started it. You must stay logged on.
  • A GEODMS_EXE set before ScheduleUpdateAll.cmd goes to the task. Other environment variables, such as GEODMS_directories_SourceDataDir, do not. The source data folder must then be in Settings > Config Options.
  • The task writes its start, its engine and finally its exit code to batch\log\UpdateAll_status.txt. Then it deletes itself.
  • Only one such task can exist at a time.
  • Windows stops a scheduled task after 72 hours.
  • To stop a run, close its console window. Or run schtasks /End /TN NM_PBL_UpdateAll and then schtasks /Delete /TN NM_PBL_UpdateAll /F.
  • Do not edit UpdateAll.cmd or UpdateAllTask.cmd during a run: cmd reads a running batch file line by line from disk.
More about the task
  • ScheduleUpdateAll.cmd refuses to start a second task, and warns when another GeoDmsRun is running.
  • The task starts once, at 00:00 of the day it was made. That time lies in the past, so the task never starts again by itself.
  • The task runs batch\UpdateAllTask.cmd, in the window of the user who is logged on (“interactive only”).

Hardware requirements

The model runs on Windows 10 or later, with GeoDMS 20.27.0 (see GeoDMS software).

Memory is what limits a run most. The table below gives per component the steps it needs, with the time and the memory peak of each step, for the morning rush of 2025 (Buurt2025 to Buurt2025, 450 origin blocks, 500 stop blocks, four departure moments) on a machine with 16 cores and 128 GB of memory. The steps of “all components” and “public transport” make caches that those components share, so they run once per reference date (the chains once per part of the day) and not once per component. The column Run says which run the numbers come from; the runs are described below the table.

Component Step GeoDmsRun calls Time Memory peak Run
all components sources, the OSM network 1 4 min 21 GB A
all components sources, the GTFS feed 1 4 min 3.3 GB A
WW output, one departure moment 9 24 min 9.6 GB A
CC output, one departure moment 9 19 min 7.2 GB A
Car cars, each departure moment 1 2.9 h 14 GB B
Car output, four departure moments 9 1.4 h 2.6 GB A
public transport legs, three base sets 1 3 min 7.8 GB C
public transport chains, 500 stop blocks in portions of 20 25 3.7 h 61 GB D
W_PT_W output, four departure moments 9 1.9 h 37 GB A
C_PT_W output, four departure moments 18 8.5 h 59 GB D
C_PT_C output, four departure moments 18 41 h 209 GB D
C_PT_RM_C output, four departure moments 18 6.1 h 52 GB D
C_PT_RMT_C output, four departure moments 18 11.9 h 98 GB D
C_PT_OVf_C output, four departure moments 18 6.9 h 58 GB D
MergeOutput output of the merge over WW, CC, Car and W_PT_W 45 3.3 h 3.4 GB A
  • A: the current model on GeoDMS 20.27.0, cycling over OSM, on 29 October 2026. The steps ran one after another, on a machine that had nothing else to do.
  • B: the car caches of 07:15, 07:30 and 07:45 on GeoDMS 20.27.0, with no other model run beside them. Each took between 2.8 and 2.9 hours.
  • C: the version of the model of 26 October 2026 on GeoDMS 20.26.0.
  • D: the version of the model of 19 October 2026 on GeoDMS 20.24.0, cycling over the network of the Fietsersbond. Two car caches were computing beside the chain step. The outputs of the cycling components ran one at a time, with no other model run beside them.

The number of calls of an output step follows from BlocksPerPortion: 450 origin blocks in portions of 50 for WW, CC, Car and the public transport components that walk to the first stop, of 25 for those that cycle to it, and of 10 for MergeOutput (see Model parameters).

The memory peak is the larger of the two numbers that GeoDmsRun writes at the end of its log, Highest CommitCharge and PeakLiveLarge, over all the calls of the step, with 1 GB as 1024 MB. For your own run, batch\ReportUpdateAll.py reads them from batch\log, together with the times and the error lines.

The public transport components differ most from each other. A component that cycles to the first stop needs more memory than W_PT_W, although it computes its origin blocks in portions of half the size, because cycling reaches many more stops than walking. C_PT_C, which also cycles from every stop to the destination, needs 209 GB. That is more than the machine has, so it paged, and it took 41 hours. A component that cycles only from a rail or metro stop (C_PT_RM_C, 52 GB) or from an OV-fiets location (C_PT_OVf_C, 58 GB) stays far below that. The times of run D come from an older version of the model and of GeoDMS, so take them as an indication: how much faster GeoDMS 20.27.0 computes these outputs has not been measured. Of the other public transport components, which walk to the first stop and cycle after it, or cycle to a rail, metro, tram or intercity stop, there is no complete measurement.

Two numbers in the table are easy to underestimate. The OSM network needs 21 GB while its cache is only 3 GB on disk, so the sources section is not as light as its few minutes suggest. And the four car caches of a rush hour take most of a day at about three hours each, which is half of what GeoDMS 20.26.0 needed (see GeoDMS software).

Run the output of one component at a time. Two output runs side by side made a machine with 128 GB of memory page, so that neither of them made progress. A smaller set of origins, such as one province, needs much less memory. Computing all blocks in one request instead of in portions also needs much more: around 100 GB for a public transport component (2023, Buurt2023 to Buurt2023).

How to lower the memory peak

The chain step and the output step are split into blocks to limit the peak: the stop blocks of the chain cache (Advanced/MaxStopBlockSize, Advanced/NumberOfStopBlocks) and the origin blocks of the output (Advanced/MaxOrgBlockSize, Advanced/NumberOfOrgBlocks). UpdateAll.cmd computes each step in portions of blocks, with a GeoDmsRun per portion. Smaller blocks or portions lower the peak, but the computation takes longer. The number of blocks per portion of the output is Advanced/Components/BlocksPerPortion (see Model parameters).

Keep at least 350 GB of disk space free for the source data, the caches and the output. The chain cache of one part of the day is about 20 GB and a car cache about 12 GB per departure moment. The GTFS and OSM caches of one reference date are 3.8 GB and 3.1 GB.

More processor cores make the chain step faster.

Table of contents