Module 4, lesson 5 · 35 minutes
The software landscape, and how to read a simulation study
The main crowd simulation packages, and ten questions to ask of any simulation claim.
You’ll be able to
- name the main packages and who makes them;
- apply a ten-question checklist to a simulation claim.
Introduction
A consultant’s report says: “Our simulation shows the new exit layout cuts evacuation time from 7 to 5 minutes.” It is a confident sentence about a model you have never seen. This lesson gives you the map of the software and the questions that turn the sentence into something you can judge.
The main packages
| Package | Developer | Notes |
|---|---|---|
| buildingEXODUS | FSEG, University of Greenwich (founded by Ed Galea, 1986) | Suite also includes SMARTFIRE, airEXODUS, maritimeEXODUS, railEXODUS |
| Pathfinder | Thunderhead Engineering | Reported “Steering” and “SFPE” modes |
| MassMotion | Oasys, Arup’s software house | – |
| LEGION | Bentley Systems | – |
| PTV Viswalk | PTV Group | Reported to be social-force based |
| Vadere | Munich University of Applied Sciences (Köster group) | Open source |
| JuPedSim | Forschungszentrum Jülich | Open source, LGPLv3 |
| Menge | Curtis, Best, Manocha | Open source, Apache-2.0 |
| SimWalk | Savannah Simulations AG | – |
| Myriad II | Still / Crowd Dynamics Ltd | Current status unverified |
The UK Cabinet Office reviewed simulation tools in 2009 in Understanding Crowd Behaviours: Simulation Tools
Ten questions for any simulation claim
- What question was the model built to answer?
- Which model class, which software, which version?
- Which fundamental diagram does it reproduce, and from which data?
- What was it calibrated against, and is that the same kind of crowd and space?
- Which verification and validation tests did it pass (ISO 20414
)? - How sensitive are the results to uncertain inputs: relaxation time, repulsion strength and range, pre-movement time?
- How many runs, and what is the spread? In our social force sandbox, three runs of the same scenario differed by up to 19 s.
- Is the output the decision metric: total time, 90% time, peak density, time above a threshold?
- What was not modelled: pushing, groups, information, counterflow, people waiting before they move?
- Would the conclusion survive if the weakest assumption were wrong?
Apply them to the consultant’s sentence. “7 to 5 minutes” from one run each could sit inside the scatter a chaotic model produces (question 7). If the new layout adds a merge, did the model represent merging (question 9)? A two-minute saving is worth having only if it survives questions 3, 6 and 7.
Which question would have exposed the social force sandbox’s “room never empties” result at 0.6 m/s?
Question 8: total time is fragile, because one person held by the door frame’s repulsion never leaves, while the 90% time is robust. Question 10 also applies: the conclusion depends on a modelling artefact.
- “Commercial software is validated software.” Ask which tests, against which data.
- “A smooth animation means a good model.” Visual plausibility is not validation.