Behavioral Validation — Västberg §4.4

HH θ̂ · 3120 workers · R=200 reps/person · 1,478,000 simulated paths · 2026-07-19

§0 — Goodness-of-fit headline

0.9126

McFadden ρ²

0.9125

Adjusted ρ̄²

Free params

3,120

Workers

200

Reps/person

Excellent fit — ρ² ≥ 0.25 (Ben-Akiva & Lerman 1985).

Quantity	Value	Notes
LL(θ̂)	-69,967.902	At estimated parameters
LL(0)	-800,278.982	Null: estimated params = 0 (HH MNL ASCs fixed)
ρ² = 1 − LL(θ̂)/LL(0)	0.9126	McFadden (1974)
ρ̄² = 1 − (LL(θ̂)−K)/LL(0)	0.9125	K=23 free params

LL(θ̂) and LL(0) are evaluated over all 7390 persons (26 groups, workers + non-workers), matching the mixed estimation sample, so the MLE property LL(θ̂) ≥ LL(0) holds. The null sets all estimated params (δ, α, β, β₀_shop, β₁_shop, β₀_leis, β₁_leis, θ_travel) to 0 while keeping the fixed HH MNL mode constants (ASCs, β_time, β_cost). It is not the uniform-path null.

§1 — Mode & activity shares

Blue = observed diaries (worker persons only); Orange = simulated under θ̂ (R=200 replications per worker). Mode shares = % of travel trips (per leg, count-weighted on both sides). Activity time shares exclude TRAVEL from the denominator on both sides — TRAVEL is the move-action between activity-states, not an activity itself; it is reported separately as travel-share (% of all 15-min slots).

Mode shares (% of travel trips, per leg)

Mode	Obs	Sim	Δ
CAR	71.8%	82.9%	+11.0%
TRAIN	0.9%	0.2%	-0.7%
BUS	2.2%	1.7%	-0.5%
WALK	17.2%	9.0%	-8.1%
BICYCLE	7.9%	6.3%	-1.7%

Activity time shares (TRAVEL excluded from denominator)

TRAVEL excluded from denominator on both sides — it is the move-action, not an activity-state. Reported separately as travel-share below.

Activity	Obs	Sim	Δ
HOME	72.6%	68.9%	-3.7%
WORK	21.7%	21.0%	-0.8%
SHOPPING	1.8%	3.5%	+1.8%
LEISURE	4.0%	6.6%	+2.7%

Travel time share

Statistic	Obs	Sim	Δ
TRAVEL (of all 15-min slots)	4.7%	4.5%	-0.2%

§2 — Activity start-time distributions (Västberg Fig 1)

Distribution of the start time of each activity episode, by hour of day. Orange bar = simulated count; blue line = observed (scaled to sim peak). Dashed lines mark means.

§3 — Trips & tours per person-day (Västberg Fig 2)

Distributions of trips/day, tours/day, and trips/tour. Orange bar = simulated; blue line = observed (scaled to sim peak).

§4 — Distribution of Trip Lengths by Mode (Västberg Fig 3)

Solid blue = observation; dashed dark red = simulation. Y-axis: #trips / #individuals. Method: 1-minute bins → 5-minute moving-average (Västberg 2020 §4.4). Observed: trip_data_higashihiroshima.csv (arrtime − deptime). Simulated: EndTime − StartTime of TRAVEL rows (15-min grid → step-discretised).

§5 — Aggregate statistics (Västberg Table 4)

Per-person scalar averages over the 3120-worker sample. LL(θ̂) and ρ² are computed over all 7390 persons (26 groups incl. 2 non-worker groups) — the same mixed sample used in estimation — so that the MLE property LL(θ̂) ≥ LL(0) holds. Behavioral figures (§1–§4) are worker-only.

Statistic	Observed	Simulated	Δ
Trips / person	2.43	1.64	-0.79
Tours / person	1.07	0.54	-0.52
Trips / tour	2.28	3.05	+0.77
Avg travel time	22.02 min	29.52 min	+7.50 min
WORK start (h)	8.42h	9.13h	+0.71h
TRAVEL start (h)	12.90h	14.82h	+1.91h

Green |Δ| < 0.2 unit; red |Δ| > 1.0 unit.

§5.5 — Simulation-fidelity diagnostic (work schedule pull)

Diagnostic: does the model correctly simulate each worker doing their scheduled work, or does logit noise overwhelm the schedule pull δ=0.03814? A residual gap between sim work hours and scheduled hours after this check is F5 — genuine model misfit (weak δ or absent activity constants), not a simulation or data-loading bug.

For each worker group: scheduled hours = end−start of work window; simulated work hours = mean over R replications; any-work fraction = fraction of reps that included ≥1 WORK step. Ratio > 0.85 is green; < 0.50 is red (indicates weak schedule pull δ or excessive logit noise).

Group	Paths (N×R)	Sched h	Sim mean h	Ratio	Any-work %
((), (), True)	218,800	—	0.0h	—	0%
((), (), False)	635,200	—	0.0h	—	0%
(('WORK',), (('WORK', 480, 1200),), True)	28,800	12.0h	12.3h	1.02	100%
(('WORK',), (('WORK', 480, 1200),), False)	19,200	12.0h	12.1h	1.01	100%
(('WORK',), (('WORK', 630, 1200),), True)	43,400	9.5h	9.5h	1.00	100%
(('WORK',), (('WORK', 630, 1200),), False)	48,000	9.5h	9.6h	1.01	100%
(('WORK',), (('WORK', 480, 1080),), True)	21,800	10.0h	10.3h	1.03	100%
(('WORK',), (('WORK', 480, 1080),), False)	26,200	10.0h	10.2h	1.02	100%
(('WORK',), (('WORK', 420, 1080),), True)	16,800	11.0h	11.3h	1.03	100%
(('WORK',), (('WORK', 420, 1080),), False)	21,200	11.0h	11.2h	1.02	100%
(('WORK',), (('WORK', 450, 840),), True)	4,000	6.5h	6.7h	1.03	100%
(('WORK',), (('WORK', 450, 840),), False)	50,800	6.5h	6.7h	1.03	100%
(('WORK',), (('WORK', 450, 1200),), True)	28,600	12.5h	12.8h	1.02	100%
(('WORK',), (('WORK', 450, 1200),), False)	22,600	12.5h	12.7h	1.02	100%
(('WORK',), (('WORK', 570, 990),), True)	8,600	7.0h	7.1h	1.02	100%
(('WORK',), (('WORK', 570, 990),), False)	39,800	7.0h	7.2h	1.03	100%
(('WORK',), (('WORK', 540, 1080),), True)	16,800	9.0h	9.2h	1.02	100%
(('WORK',), (('WORK', 540, 1080),), False)	44,600	9.0h	9.2h	1.03	100%
(('WORK',), (('WORK', 570, 810),), True)	7,600	4.0h	4.3h	1.07	100%
(('WORK',), (('WORK', 570, 810),), False)	43,400	4.0h	4.4h	1.10	100%
(('WORK',), (('WORK', 480, 990),), True)	4,600	8.5h	8.8h	1.04	100%
(('WORK',), (('WORK', 480, 990),), False)	33,800	8.5h	8.8h	1.03	100%
(('WORK',), (('WORK', 450, 990),), True)	9,600	9.0h	9.3h	1.04	100%
(('WORK',), (('WORK', 450, 990),), False)	43,600	9.0h	9.2h	1.03	100%
(('WORK',), (('WORK', 480, 840),), True)	3,000	6.0h	6.2h	1.04	100%
(('WORK',), (('WORK', 480, 840),), False)	37,200	6.0h	6.3h	1.04	100%

§6 — Measurement vs Model: what was fixed vs what remains

F1–F4 (measurement asymmetries — fixed in this report)

F1 — CAR_PASS folding: simulated car-passenger trips were not folded into 'CAR' before comparison. Fixed: CAR_PASS→CAR on sim side everywhere. Net effect: sim CAR share was understated by ~30pp vs the folded figure.
F2 — Mode-share weighting: observed mode shares were time-weighted (per 15-min in-transit slot) while simulated were count-weighted (per trip leg). Fixed: both sides now count trips per leg.
F3 — TRAVEL in activity-share denominator: TRAVEL was in the observed denominator (5.1%) but not in the simulated denominator — inflated sim HOME/WORK shares. Fixed: TRAVEL excluded on both sides; reported separately as travel-share.
F4 — TRAVEL timing asymmetry: simulated TRAVEL timing was departure-time spikes (1 row per leg); observed was per-15-min in-transit occupancy. Fixed: sim TRAVEL legs expanded to per-slot occupancy before building timing distribution.
F6 — Activity-map silently dropped 'Other' (891 slots): old code had no mapping for 'Other' → slots dropped. Fixed: uses data_loader.ACTIVITY_MAP which maps 'Other' and 'Sightseeing' → LEISURE, consistent with how estimation loaded the data.

F0 — McFadden ρ² sample discipline (fixed)

The original ρ² = −0.864 was computed on a worker-only subset (3,025p) while θ̂ was the MLE on the full mixed sample (5,218p workers+non-workers). The MLE guarantee LL(θ̂) ≥ LL(0) only holds on the estimation sample. Additionally, LL(0) was computed on a separately rebuilt graph, risking state-index misalignment. Both are fixed: LL(θ̂) and LL(0) now use all 7390 persons + the same graph-state indices (recompute_graph_utilities on the existing graph object, not rebuild).

F5 — Genuine model misfit (structural): activity shares

After all F1–F4 fixes, a gap remains: simulated WORK time share ≈21.0% vs observed 21.7% (-0.8pp gap). This is not a measurement artifact — it is a structural property of the K=10 "Option A" specification:

No activity alternative-specific constants (ASCs) — Västberg (2020) §4.4 explicitly estimates these (activity ASCs, tour ASCs, trip ASCs) specifically to force the simulated marginal distributions (trips/day, mode split, work share) to match observed. Our K=10 spec fixes the mode ASCs at HH MNL estimates and has no activity ASCs.
A model without activity constants has no mechanism to match activity-time-share marginals. Any residual gap is therefore expected and not attributable to a code bug.
The schedule pull δ=0.03814/min is the only force directing workers toward WORK; β₀_leis=0.000 and β₀_shop=0.000 make discretionary activities less attractive than HOME; θ_travel=0.221 halves commute disutility; logit noise competes against δ at every step.

F5b — Genuine model misfit (structural): transit collapse

Simulated transit mode share ≈ 0% vs observed ≈ 3.4% (BUS + TRAIN). This is not a code bug or OD-data gap; it is a quantified consequence of the K=10 spec. Diagnostic run estimation/experiments/mode_collapse_diagnostic.py establishes the following:

Mode	ASC (raw)	Effective ASC (×θ=0.408)	Median TT	u_travel (per trip)
CAR	+2.756	+1.124	30 min	+0.54 (positive — ASC beats time+cost)
BUS	0.000	0.000	36 min	−0.81 (negative — no ASC to offset travel penalty)
TRAIN	0.000	0.000	50 min (0.4% feasible)	−1.04 (rare service + no ASC)

TRAIN: 99.6% of OD pairs have TT=0 (no service) — Higashihiroshima has very limited rail coverage. TRAIN is structurally infeasible for virtually all trips.
BUS: 99.4% feasible but utility is always negative — ASC=0 means the travel time penalty is never offset by a mode preference. BUS is only chosen when the destination activity is worth at least +0.81 utiles/step extra — exceeding the WORK schedule pull (δ=0.015/min × 15 min = 0.23/step).
CAR has a permanent +1.124 advantage over PT after θ_travel scaling. To make BUS competitive with CAR requires ASC_pt_adj ≈ +1.35 utiles (≈ +3.32 in raw K=11 units). This is the parameter retired in Option A.

The "θ_travel lifts PT share" claim in the code comment is directionally correct — at θ_travel=1.0 (full MNL), CAR ASC=2.756; at θ_travel=0.408 it is 1.124 — but the absolute PT share remains near zero because BUS still starts at 0 and has negative total utility on median trips.

Decision implication (mode): matching the observed 3.4% transit share requires at minimum a free PT constant (one extra param, "Option B" / K=11 lite) or full activity/mode ASC estimation (Västberg §4.4). See §7.

Decision implication (activity shares): to match activity-time-share marginals, the model spec needs activity-/tour-/trip-level ASCs estimated jointly with the DP parameters. That is a model-spec change; it is not in scope of this validation.

§7 — Caveats and scope

Mode spec options: to improve the transit / mode-share fit, three options exist: A) document as known K=10 Option A limitation (current); B) add a single free PT constant (ASC_pt_adj ≈ +3.32 in MNL units) — makes K=11, no re-derivation of gradient; C) estimate full activity/mode/tour ASCs jointly with DP parameters (Västberg 2020 §4.4). Options B and C require a re-estimation run; they are not in scope of this validation.
ρ² discipline: ρ² = 1 − LL(θ̂)/LL(0) is evaluated on the same 7390-person mixed sample as estimation, using the same graph-state indices. LL(0) sets the 8 estimated params to 0; fixed HH MNL mode constants (ASC, β_time, β_cost) are unchanged. ρ² is valid (the MLE guarantee LL(θ̂) ≥ LL(0) holds on this sample).
Same-subsample discipline: observed behavioral stats (§1–§4) are restricted to the 3120 worker persons only, since the behavioral figures (work timing, trip counts) only make sense for workers with a schedule. ρ² and LL use all 7390 persons. Motorcycle/Taxi persons and persons with unmodelled mandatory activities (Hospital, CHILD_DROPOFF) are excluded.
'Other' activity mapping: 'Other' and 'Sightseeing' observed slots (n_dropped_act_slots from disabled Hospital/Pickup) are mapped to LEISURE via data_loader.ACTIVITY_MAP, consistent with how the estimation loaded the data. Disabled-activity slots are excluded.
15-min step coarseness: simulated trip durations are multiples of 15 min. Observed durations are exact (minute-level). Lower-end trip-length distributions differ by construction.
Observed trip length: obs_trip_len uses the 3120-worker subset (same as simulation).
Tour segmentation: observed tours from trip_data_higashihiroshima.csv (purpose='Home' splits tours); simulated tours from TRAVEL/HOME alternation. Small definitional gap.
In-sample only: this validates the same persons used for estimation. Out-of-sample validation requires a held-out subsample.
car_owned grouping: non-owner persons are assigned CAR-excluded graphs (matching estimation); CAR_PASS is available as a mode to passengers even in non-owner groups (passenger of someone else's car), and is folded into CAR for comparison.