DC ParaTransit Info has been asked why we don’t trust the statistics WMATA publishes on their web site. Outside of WMATA’s exclusion of complaints filed in the most logical place for riders to complain to, the simple fact is, their numbers literally don’t add up.
We know that WMATA disclaims their figures as being preliminary, but even their preliminary figures don’t add up, even allowing for rounding errors. We ran basic math on their published figures, and easily found these discrepancies in the stated on-time performance total, based on the total statistics WMATA is making public:
| Service Date | On-time | Trips Delivered | Late Trips | Excessively late trips | Our on-time calc. | Difference between WMATA on-time calc. and ours | Unaccounted for trips |
|---|---|---|---|---|---|---|---|
| Tue, February 21, 2006 | 91.60% | 4165 | 380 | 101 | 88.45% | 3.15% | 131 |
| Sat, February 18, 2006 | 89.30% | 1458 | 110 | 48 | 89.16% | 0.14% | 2 |
| Fri, February 10, 2006 | 91.10% | 4306 | 290 | 101 | 90.92% | 0.18% | 8 |
| Thu, February 9, 2006 | 91.10% | 4321 | 279 | 114 | 90.90% | 0.20% | 8 |
| Wed, February 8, 2006 | 91.10% | 4508 | 292 | 122 | 90.82% | 0.28% | 13 |
| Mon, February 6, 2006 | 92.50% | 4383 | 240 | 125 | 91.67% | 0.83% | 36 |
| Sun, February 5, 2006 | 89.80% | 1118 | 86 | 76 | 85.51% | 4.29% | 48 |
| Sat, February 4, 2006 | 88.40% | 1563 | 139 | 52 | 87.78% | 0.62% | 10 |
| Fri, February 3, 2006 | 91.30% | 4439 | 311 | 121 | 90.27% | 1.03% | 46 |
| Tue, January 31, 2006 | 90.70% | 4575 | 357 | 72 | 90.62% | 0.08% | 4 |
| Mon, January 30, 2006 | 92.40% | 4749 | 363 | 121 | 89.81% | 2.59% | 123 |
| Sun, January 29, 2006 | 91.80% | 1205 | 99 | 26 | 89.63% | 2.17% | 26 |
| Sat, January 28, 2006 | 88.20% | 1739 | 205 | 44 | 85.68% | 2.52% | 44 |
| Fri, January 27, 2006 | 87.50% | 4561 | 570 | 172 | 83.73% | 3.77% | 172 |
The MetroAccess computer should be able to produce at the end of the day the total number of trips, and the number of trips that were late pick-ups. Those figures, posted by WMATA, should be solid. Given those figures, it should be a simple matter to calculate the on-time performance (total number of trips – number of late trips – number of excessively late trips divided by the total number of trips).
When DC Paratransit Info plugged WMATA’s stated figures into a spreadsheet and ran that calculation, we found that most of the statistics, allowing for rounding errors, did come within 0.10% of the on-time performance WMATA was giving – except for those dates above. In those cases, the math gives us much more different results than WMATA is giving.
We don’t know why the simple math to get the performance data doesn’t correspond to WMATA’s figures. We only know that we end up with “missing” trips from the totals as a result of trying to reconcile their trip data with their performance estimate.
There could be several reasons why the numbers don’t add up, all of them innocuous.
- There may be a criteria in the calculation that WMATA isn’t telling us about (although it would seem obvious that on-time performance would be a simple expression of total trips – late trips).
- There may be data entry errors in the figures being published.
- WMATA might be posting the “reconciled” on-time performance figure but not bothering to update the rest of the numbers for a day.
Whatever the reasoning, it highlights the fact that we cannot rely on the published figures WMATA is making available. If WMATA is applying other factors to their calculation that they are not publicizing, it makes posting the figures as they have been a pointless exercise, because they are incomplete, making it impossible to independantly verify even the math WMATA is using.
WMATA is also, as far as we can tell, only posting trips completed. There is no mention of any statistics for trips that MetroAccess does not complete, either because of rider no-shows, or more importantly MetroAccess no-shows. Logically, the MetroAccess no-shows would be considered “excessively late”, but we have no way of knowing if they’re being included. Could this be the source of the “missing” trips?
WMATA should be commended for starting to post the service performance figures. But the figures on each row should match up. Without knowing why they don’t, they become a pointless exercise, and untrustworthy. If WMATA is serious about releasing data that proves the service is improving, it should provide accurate data, and the context to evaluate that data reliably.
Until then, we have no alternative but to continue to look on WMATA’s statistics as little more than a PR exercise.
Sphere: Related Content
