Validating the Pareto front

Two scripts compare the public transport output of two runs that differ only in MinimiseCriteria (#53). Run A leaves out the price and is the reference; run B has it. If B is right, its front contains the fastest option of A, at no higher price, and no row of B is dominated by another row of the same pair.

The reference leaves out only the price: 'Time,Car,Bike', the default of -CritA, keeps per pair the fastest option, the fastest without a car, the fastest without a bicycle and the fastest without both. With 'Time' alone, car and bicycle ownership are not criteria either (Car and bicycle ownership as criteria applies only when Car or Bike is named), and the final selection keeps only the fastest option of each pair, as if every traveller had a car and a bicycle. In a component that needs neither a car nor one’s own bicycle, such as W_PT_W, the two give the same result.

Running the comparison 

powershell -ExecutionPolicy Bypass -File batch\RunFrontValidation.ps1 -Label val53_tcb -BaseRun C:\LocalData\NM_PBL_runs\<run>\NetworkModel_PBL
Parameter Default Meaning
-Label (required) Name of the comparison: the run copies go to C:\LocalData\NM_PBL_runs\<Label> (-RunsDir), the output of a run to Output\<Analysis_date>_<Label>_A or _B, with the suffix of the component. Letters, digits, - and _, at most 18 characters.
-BaseRun none An existing run copy (the folder with cfg, data and batch) whose output serves as B. Only A is computed.
-CritA 'Time,Car,Bike' MinimiseCriteria of run A.
-CritB, -AnalysisMoment, -PartOfDay, -Component 'Price,Time,Car,Bike', Y2026, Ochtendspits, W_PT_W Run B, and the reference date, part of the day and component of both runs, without -BaseRun; with it they come from the base run. The component is WW, CC, Car or <pre>_PT_<post>, which the script sets as Component and, for public transport, PreTransport and PostTransport.
-Blocks, -NrOrgBlocks (1..10), 450 The origin blocks whose output is computed and compared.
-WaitFor base with -BaseRun, else others Before each heavy step: base waits until the base run has finished, others until no GeoDmsRun of another NetworkModel_PBL run is running, none does not wait.
-MinFreeRamGB 24 Also wait while less memory is free. The script stops when C: has less than 15 GB free.
-Exe C:\Program Files\ObjectVision\GeoDms20.27.0.m\GeoDmsRun.exe The GeoDmsRun.exe of the runs; pass one that meets GeoDMS software.
-RunsDir, -OutRoot C:\LocalData\NM_PBL_runs, C:\LocalData\NetworkModel_PBL\Output Where the run copies go, and the Output folder of the model (%LocalDataProjDir%\Output).
-DryRun   Only make the run copies and show the parameters that were set.

batch\ValidateFronts.py reads only the csv files per block, so both runs need OutputFormat 'csv'. RunFrontValidation.ps1 does not set it: with -BaseRun the base run must have written csv, and without it the copies take OutputFormat from HEAD.

With -BaseRun the script:

  1. reads AnalysisMoment (with the analysis day and the feed of its preset), MinimiseCriteria, OutputLabel and the component of the base run, waits until the base run has finished, and copies its output for -Blocks and its signature.txt to <Label>\B_uitvoer, so that a later run under the label of the base run cannot change B;
  2. copies cfg, data and batch of the base run (without batch\log) to <Label>\A\NetworkModel_PBL, so that run A has the same commit, feed, date and ConfigSettings.dms as B. In the copy it sets only MinimiseCriteria to -CritA, OutputLabel to <Label>_A and Export_PriceInformation to TRUE;
  3. runs batch\UpdateAll.cmd chains in that copy. The price criterion is part of the chain cache signature, so A gets its own chain cache; Car and Bike do not change the chain cache;
  4. computes the output of the origin blocks (Block_<i>of<N>/PublicTransport/Generate_Output/Traveltime_ForEachDepTime_Fence/Generate);
  5. compares the signatures of A and B without the criteria (MinCrit-..._MaxPT and _min-..._maxtransf) and writes LET OP to the status file when they differ in anything else;
  6. runs batch\ValidateFronts.py on the two outputs.

Without -BaseRun it makes two copies of HEAD with git archive, sets AnalysisMoment, PartOfDay, the component, the criteria, the labels and Export_PriceInformation, and computes the chains and the output of both. Progress is written to <Label>\status.txt and the report to <Label>\verslag.md.

The chain step of a run takes hours (about 4 hours for 500 stop blocks in 2026), the output of 10 origin blocks about 10 minutes and the comparison about a minute. The script should therefore not depend on the window it was started from; start it detached, for example

Start-Process powershell -WindowStyle Hidden -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-File','batch\RunFrontValidation.ps1','-Label','val53_tcb','-BaseRun','C:\LocalData\NM_PBL_runs\<run>\NetworkModel_PBL'

The checks

python batch\ValidateFronts.py --a "<glob A>" --b "<glob B>" [--blocks 1-10] [--moments 07h00m] [--eps-time 1.0] [--eps-price 0.10] [--md report.md] can also be run on its own. It pairs the csv files per block of A and B on departure moment and block, reads the criteria of each run from signature.txt in its output folder, and reads the columns Traveltime_<moment>, ModeUsed_<moment> and, when present, Price_<moment>, NeedsCar_<moment> and NeedsBike_<moment>. The price checks fall away when a run has no Price column. It checks per origin-destination pair and departure moment:

Check Meaning
1 The shortest travel time of B equals that of A, within --eps-time (default 1 minute).
2 The price at the shortest travel time of B is not higher than in A, within --eps-price (default 0.10 euro). Equality cannot be required: a run without price picks any price among equally fast options.
3, 4 Within B the shortest time is at most the time at the lowest price, and the price at the shortest time at least the lowest price. True by definition; a violation points to missing values.
5 No row of B is dominated by another row of the same pair on time, price and the ownership criteria of B; counted strictly and beyond the epsilons.
6 Pairs that occur only in A or only in B.

Checks 1, 2 and 6 are made over all options and over the public transport chains alone and, when A and B both name Car or Bike, also per group of travellers: without a car (NeedsCar 0), without a bicycle (NeedsBike 0) and without both. A Boolean criterion has no epsilon, so the fastest option of such a group in A must come back in B; a pair of a group that is only in A is an error there.

Results

As an example, 'Time' against 'Price,Time' for origin blocks 1 to 10 (feed 20260925, analysis day 6 October 2026) passed checks 1, 2 and 5 for all 1,306,535 combinations of pair and departure moment. 25 public transport chains were only in A; each was replaced in B by a cycling or car trip that was slightly slower and cheaper, which the epsilon allows. Neither run named Car or Bike, so these chains disappear only in that set-up. See #53.