text:
October 2007
a publication of the
|1. Report No. FHWA/CA/OR-2006/31
|2. Government Accession No.
|3. Recipient's Catalog No.
|4. Title and Subtitle Beyond Uncertainty: Modeling Transportation, Land Use, and Air Quality in Planning
|5. Report Date October 2007
|6. Performing Organization Code
|7. Authors Caroline J. Rodier, Ph.D.
|8. Performing Organization Report No. 07-01
|9. Performing Organization Name and Address Mineta Transportation Institute College of Business San José State University San José, CA 95192-0219
|10. Work Unit No.
|11. Contract or Grant No. 65W136
|12. Sponsoring Agency Name and Address California Department of Transportation Sacramento, CA 95815 U.S. Department of Transportation Research and Innovative Technology Administration 1200 New Jersey Avenue, SE Washington, DC 20590-0001
|13. Type of Report and Period Covered
|14. Sponsoring Agency Code
|15. Supplementary Notes
|16. Abstract Concerned citizens across the United States are increasingly asking officials about the effects of proposed new highways and their alternatives, such as transit and road pricing, on how their communities will grow, the air their children will breathe, and the amount of time they will have to spend in traffic commuting to work. It is widely acknowledged, however, that the models used to assess these effects have limited accuracy and sensitivity to alternatives to highway expansion. This study attempts to move beyond the issues of uncertainty in models used to forecast the travel, land use, and air quality effects of transportation projects and policies by (1) reviewing the literature on error and uncertainty in travel and land use models to understand key sources, likely confidence bounds, and potential biases; (2) conducting interviews with modeling experts to gain insight into how uncertain models may be improved and better applied in transportation studies; and (3) presenting a series of cases studies that illustrate innovative and, possibly, more credible approaches to modeling given different study objectives, model capability, and knowledge of model uncertainty.
|17. Keywords Air quality; Land use models; Land use planning; Policy analysis; Urban planning
|18. Distribution Statement No restriction. This document is available to the public through the National Technical Information Service, Springfield, VA 22161
|19. Security Classif. (of this report) Unclassified
|20. Security Classif. (of this page) Unclassified
|21. No. of Pages 60
|22. Price $15.00
All rights reserved
Library of Congress Catalog Card Number: 2007937691
To order this publication, please contact the following:
Mineta Transportation Institute
College of Business
San José State University
San José, CA 95192-0219
Tel (408) 924-7560
Fax (408) 924-7565
--> >http://transweb.sjsu.edu
The author would like to thank the Mineta Transportation Institute and the California Department of Transportation for funding this research. In particular, thanks goes to Trixie Johnson of the Mineta Transportation Institute for her continued support of research on issues related to transportation modeling and to J.M. Pogodzinski for his comments on the draft report. Thank you also to Megan Smirti for her assistance with the expert interviews and to all the experts who gave their time to participate in the interviews conducted for this study. The contents of this report reflect the views of the author, who is responsible for the facts and the accuracy of the data presented herein.
Thanks are offered also to MTI staff, including Research and Publications Assistant Sonya Cardenas, Webmaster Barney Murray, and Graphic Artist Shun Nelson. Editing and publication services were provided by Catherine Frazier and Irene Rush.
Forecast Versus Actual Performance of New Projects
Model Validation Tests
Sensitivity Analyses of Model Input and Parameter Errors
Sensitivity Analyses of Models' Theoretical Validity
Expert Interviews
Conclusions and Recommendations
Error Analyses for More Credible Regulatory Modeling
"Visioning" with Improved Models without Error Analyses
Error Analyses to Specify Evaluation Criteria for "Visioning"
Application of Unimproved Models That Address Induced Travel
Future Directions in Modeling under Conditions of Uncertainty
Sensitivity Scenarios
Alternative Scenario Results
1. A Comparison of Forecast Versus Actual Boardings for New Light Rail Projects from Richmond, 1998
2. Summary of Tests Used in the Validation Studies
3. Summary of Validation Study Results
4. Univariate Sensitivity Analyses
5. Summary of Multivariant Sensitivity Analyses
6. Model Components and Variables in the Case Study Models
7. An Example of Sensitivity Tests
8. Long-Term Induced Travel Sensitivity Test Results with the Case Study Models
9. Summary of Core Study Policies
10. Summary of Scenarios for Sensitivity Analyses
11. Percentage Change in Acres of Land Consumed by Superzone in the 2020 Sensitivity Scenarios Compared to the Base Case Scenario
12. Percentage Change in Daily Travel and Emissions Results in the 2020 Sensitivity Scenarios Compared to the Base Case Scenarios
13. Percentage Change in 2020 Household and Employment Activities Compared to the Base Case Scenario
14. Percentage Change in 2020 Land Consumption (Acres) Compared to the Base Case Scenario
15. Percentage Change in 2020 Daily Mode Share Results Compared to the Base Case Scenario
16. Percentage Change in 2020 Daily Vehicle Travel Results Compared to the Base Case Scenario
17. Percentage Change in 2020 Daily Emissions (Tons) Results Compared to the Base Case Scenario
Concerned citizens across the United States increasingly are asking officials about the effects of proposed new highways and their alternatives, such as transit and road pricing, on how their communities will grow, the air their children will breathe, and the amount of time they will have to spend in traffic commuting to work. It is widely acknowledged, however, that the models used to assess these effects have limited accuracy and sensitivity to alternatives to highway expansion. This study attempts to move beyond the issues of uncertainty in models used to forecast the travel, land use, and air quality effects of transportation projects and policies by
In "Synthesis of the Literature on Model Error and Uncertainty," the range of plausible errors in transportation project and policy studies is identified based on three types of studies: forecast versus actual project performance, model validation tests, and model sensitivity analyses. Each study type has its strengths and weaknesses, but cumulatively, these results begin to delineate reasonable confidence bounds for models typically used in transportation planning.
The results of the most comprehensive study comparing forecasts and actual project use suggest the following: 1
The review of the literature also provides insight into the relative magnitude of various sources of model error. The evidence for travel demand models suggests that population projections are key sources of error and uncertainty. 9 It also appears that the effect of sources of errors are more specific to the structure of land use models than travel demand models. For example, population projections were important sources of error in the Sacramento MEPLAN model and the Eugene-Springfield UrbanSim model, but were less significant in the Austin DRAM-EMPAL model.
Finally, the literature review addressed the ability of current models to represent induced travel and the magnitude of possible biases in the evaluation of alternative scenarios when a model does not represent induced travel effects. Induced travel is defined as how a change in transportation system supply changes the time and monetary cost of travel, and thus the total travel demand, all else being equal. The results of the validation studies of the Sacramento travel model and the MEPLAN integrated land use model suggest that both models underestimate induced travel, but the relative magnitude of the error was greater in the travel model than in the integrated land use model. 10 Other sensitivity studies suggest that models are able to represent induced travel within the range documented in the empirical literature, if travel times are represented consistently throughout the model hierarchy. 11 If induced travel is not represented, the need for and benefit of the roadway alternative, relative to a no-build alternative, will be overstated and negative emission effects will be understated. 12
In "What the Experts Have to Say: Improving and Applying Uncertain Models in Transportation Planning," the results of a series of interviews conducted with travel and land use modeling experts are presented and provide insight into how uncertain models may be improved and better applied in policy studies. Many of these experts felt strongly that the uncertainty in any particular modeling analysis should be made as explicit as possible by presenting results with confidence intervals rather than point estimates. They also indicated that modeling a range of alternatives to identify which investments and policies would perform best, under different and uncertain conditions, was a good way to deal with modeling uncertainty. Many also stated that the use of more advanced activity-based microsimulation modeling tools would facilitate such uncertainty analyses. Experts further recommended that there be a greater investment of public dollars or more effective use of public dollars to provide incentives for model improvement and to develop effective processes of model auditing and oversight.
"Some Innovative Approaches to Modeling Under Uncertainty,"
case studies are used to illustrate innovative modeling approaches and future directions given the objective of the study, model capability, and knowledge of model uncertainty. They include how error analyses can be applied to regulatory modeling; how improved models can be used to help communities envision alternative transportation and land use futures with and without error analyses; and how even limited travel demand models can be applied to address stakeholder concerns related to highway investment and induced travel.
Concerned citizens across the United States increasingly are asking officials about the effects of proposed new highways and their alternatives, such as transit and road pricing, on how their communities will grow, the air their children will breathe, and the amount of time they will have to spend in traffic commuting to work. It is widely acknowledged, however, that the models used to assess these effects have limited accuracy and sensitivity to alternatives to highway expansion. This study begins with a review of the literature on error and uncertainty in travel and land use models that identified key sources of errors, likely confidence bounds, and potential biases. Next, the results of interviews with modeling experts are reported. These results suggest how uncertain models can be improved and better applied in transportation studies. Finally, cases studies are described that illustrate innovative and, possibly, more credible approaches to modeling given different study objectives, model capability, and knowledge of model uncertainty.
In this section, the relatively large body of literature on error and uncertainty in travel and land use models is reviewed to identify a range of plausible errors for both project-level and regional modeling analyses, the relative magnitude of different sources of model errors, and the potential bias introduced by models that do not represent certain theoretical relationships.
Several studies have compared the forecast (typically by some sort of a model) and actual use of specific road and transit projects. Such analyses provide insight into the frequency, magnitude, and direction of errors in project-based analyses. These results can be applied to provide plausible confidence intervals on forecasts of proposed new transportation projects by using some reference class of projects (for example, a 95 percent confidence interval that the actual project performance will lie within ±5 percent of the forecast value). 13
The most recent and comprehensive comparison of forecast and actual use of new transportation projects was conducted by Flyvbjerg, et al. 14 This study examined 210 transportation construction projects completed between 1969 and 1998 in 14 countries, including 27 rail projects and 183 road projects. Actual traffic (number of vehicles for road projects and number of passengers for rail projects) in the first year after project completion was compared with first-year traffic as forecast at the time each project was approved. The inaccuracy of forecasts was calculated as the percentage change from actual to forecast use (forecast value is subtracted from actual value, divided by forecast value, and then multiplied by 100). The percentage change is also known as the mean algebraic percent error (MALPE). The mean estimates of inaccuracy were not weighted by project size. For each of these projects and an additional 24 projects, project managers and researchers were asked to enumerate possible factors that would explain discrepancies.
For road projects, Flyvbjerg, et al. 15 found that more than 50 percent of projects were inaccurate by more than ±20 percent and more than 25 percent of projects were inaccurate by more than ±40 percent. These forecasts were "underestimated by an average of 8.7 percent (with a 95 percent confidence interval between 3 to 15.9 percent), resulting in actual traffic that was on average 9.5 percent higher than forecast traffic (s. d. = 44.3, 95 percent confidence interval of 3.0 to 15.9)." 16 However, there was "no significant difference between the frequency of inflated versus deflated forecasts for road vehicle traffic (p = 0.822, two-sided binomial test); 21.3 percent of projects have inaccuracies below -20 percent, whereas
28.4 percent of projects have inaccuracies above +20 percent." 17 The authors also found that
the accuracy of traffic forecasts has not improved over time. Based on an analysis of 51 cases, forecasts for smaller road projects tended to have higher levels of inaccuracies than larger road projects. Planners and researchers interviewed to identify the sources of these inaccuracies identified the following elements of model forecasts as the top three: the number of trips or trip generation (27 percent), land use development (26 percent), the origin and destinations of trips or trip distribution (23 percent); nonspecific problems totaled 22 percent. (See "Sensitivity Analyses of Models' Theoretical Validity " on page 17 for further explanation of sources).
For rail projects, Flyvbjerg, et al. 18 found a negative bias in the distribution of inaccuracy: forecast use was greater than actual use for 85 percent of projects and less than actual use for only 15 percent. Sixty-seven percent of projects had a negative inaccuracy of more than 67 percent, and the average inaccuracy was -65.2 percent (with 95 percent confidence interval between 23.1 to 151.3 percent). In general, higher levels of inaccuracy were correlated with higher project cost but not with number of passengers or length of implementation. It was also found that the accuracy of forecasts for rail projects has not improved over time. The sources reported as important were the origin and destination of trips or trip distribution (29 percent), "deliberately slanted" (25 percent), and the number of trips or trip generation (11 percent). 19
A study in the United Kingdom compared forecast and actual traffic for 151 road projects (counts taken about one year after opening); the results indicated that "there was a wide discrepancy, but on average the observed traffic on the improved roads was about 10 percent higher than has been forecast." 20
Two studies in the United States examined forecast and actual use of new transit projects. Like the previous studies, these studies examined the percentage difference from actual to forecast use, and the mean error results were not weighted by project size. The first is the well-known study by Pickrell in 1990.
This study compared the original forecasts made when the project was approved to actual average weekly boardings for 10 major transit projects from 1971 to 1987. The percentage difference ranged from a low of -28 percent to a high of -83 percent (across different time horizons) with an average mean error of -67 percent. More recently, Richmond
examined the performance of 12 new light rail projects in operation as of April 1997, all of which were completely new as of that date.
Table 1
presents the results of projects for which forecast and actual boarding, consistent with those used by Pickrell, are available. The mean percentage difference is -33 percent, and the error ranged from 21 percent to -106 percent. The finding of a tendency toward overinflated forecasts of use for transit projects in both these U.S. studies is consistent with that of Flyvbjerg, et al.
Table 1 A Comparison of Forecast Versus Actual Boardings for
New Light Rail Projects from Richmond, 1998
|Percentage Difference from Actual to Forecast
|Forecast Time Horizon (Years)
|Denver Light Rail Line to Downing St.
|Portland Light Rail
|San Diego South (Blue) Line Light Rail
|San Jose Light Rail
|St. Louis Light Rail
In the typical model development process, models are estimated on local data then calibrated or adjusted to closely match observed data. Validation tests show how well model forecasts match observed data that were not used for model estimation or calibration. Validation tests can be developed and applied to test uncertainty in specific model components and an entire model set, depending on data availability. The relative realism, simplicity, and ease of communication are considered to be advantages of this approach. Validation tests can be used to bracket the uncertainty of a wide range of model applications in policy studies, including regional travel and air quality analyses and cumulative effects in environmental impact analyses. To date, the author is aware of only a few validation studies of models used in transportation and air quality planning. 24 Descriptions of these studies are summarized in Table 2 on page 12 and the study results are described in Table 3 on page 12 .
A validation study of the Eugene/Springfield (Oregon) UrbanSim land use model was conducted by Waddell 25 using an R-square measure of goodness-of-fit between the 1994 forecast versus observed employment, population, land value, and development square feet. As the level of the spatial aggregation unit of analysis increased in size, the goodness-of-fit or estimate of accuracy improved. The R-square results ranged from 0.45 to 0.64 for smaller cells and from 0.64 to 0.88 for larger zones.
Validation tests were also performed on the Sacramento, California, region's travel demand model (SACMET version 1991) by Rodier 26 over a nine-year period. Total model error was estimated by forecasting year 2000 travel with projected (in 1991) 2000 zonal demographic data and transportation network, then calculating the percentage change from forecast to observed 2000 travel (or the MALPE). Over a nine-year period, the model overestimated (total model error) vehicle miles traveled (VMT) by 11.5 percent, vehicle hours of travel (VHT) by
12.8 percent, and vehicle hours of delay (VHD) by 38.4 percent.
In this study, validation tests were also developed to identify the separate contribution of zonal demographic projections (future estimates of the number of households and employment by zones) and model error (model functional forms and parameters) to the total model error described above. It was found that each source contributed approximately equally to total model error for VMT and VHD, but for VHT, demographic input error accounted for about two-thirds of total error.
Finally, validation tests were applied to explore SACMET's representation of induced travel by holding the 1991 network constant in two separate forecasts of year 2000 travel, one with projected (in 1991) 2000 zonal demographic data and the other with observed 2000 demographic data. 27 The results of the analysis indicated that the model underestimated induced travel compared to the estimate of actual induced travel in this study by almost half (model elasticity of VMT with respect to lane miles is 0.14 model and estimated actual is 0.22).
The integrated land use and transportation model, a later version of the Sacramento MEPLAN model, was the subject of a subsequent validation study by Rodier. 28 This validation study tested errors resulting from model functional forms and calibrated parameters. Land use and travel for the year 2000 were simulated with the Sacramento MEPLAN model (calibrated to 2000 data) with the year 1990 observed household, employment, vacant land, and land developed by zone; observed regional employment and population growth from 1990 to 2000; and observed transportation networks for each model time step from 1990 to 2000. The results of this simulation were compared to available observed year 2000 data to assess model errors. Errors in zonal land use forecasts are represented by both algebraic and absolute errors. The algebraic error (ALE) was calculated as:
ALEi = F1i – O1i (1)
where F1 is the forecast year 2000 value, O1 is the observed year 2000 value, and i is a Sacramento MEPLAN zone for land use categories or regional travel category (for example, total regional mode share, distance, or time). The mean algebraic error (MALE), where n is equal to the total number of zones, was calculated as:
∑ ALEiMALE =
n (2) Next, the algebraic percent error (ALPE) was calculated as:
APLE = ⎜⎜⎛ ALE 1 i ⎟⎟⎞ *100
⎝ Oi ⎠ (3)
Finally, the mean algebraic percent error (MALPE) of the forecast value across zones was calculated as:
∑ ALPEiMALPE =
n (4) The absolute value of the ALPEi (|ALPEi|) is the absolute percent error (APEi).
In general, the results of the model error tests indicated relatively large errors in land use projections. Absolute percent errors were within zero to 25 percent for 20 percent of zones for employment and nonresidential land and for 50 percent of zones for households and residential land. Eighty percent of the zones had absolute percent errors for employment and households within zero to 75 percent and for nonresidential and residential land within zero to about 110 percent. The mean algebraic percent errors across land use categories ranged from 7 percent for employment, 54 percent for nonresidential land, 60 percent for households, and 86 percent for residential land. Most zones in the region (48 of the 71 zones for employment and 42 zones for households) had negative algebraic percent errors from -100 to zero percent. There were relatively modest errors (less than 50 percent) for the more established central urban areas. However, the model appeared to overestimate the location of households and employment in the outer areas of the region with relatively less expensive land. Possible explanations for these results include limited price sensitivity in the developer model due to limited price data used to estimate the model, and larger zones in the outer areas of the region with only one centroid connector that may underestimate travel times.
The travel forecast errors (expressed as MALPE) for the Sacramento MEPLAN model were generally less pronounced relative to land use errors. The mode share results indicated lower error levels for drive, carpool, and walk modes (11, 3, and 6 percent underestimated, respectively) and higher error levels for the transit and bike modes (39 and 105 percent overestimated, respectively). These results may be due in part to the overestimate of average vehicle travel times (by about 14 percent) and the underestimate of average vehicle travel speed (by about 4 percent). As a result, the model underestimates vehicle trips and VMT by 11 and 3 percent, respectively.
Finally, the land use and travel changes induced by the expansion of the regional transportation network from 1990 to 2000 were estimated by simulating the year 2000, holding the 1990 network constant for each future time step (1992 to 2000). The moderate roadway and highway expansion in the region simulated over the 10-year period by the Sacramento MEPLAN model also produced a reduction in average vehicle travel time
(7.6 percent) and an increase in average travel speed (15.7 percent) leading to a modest increase in vehicle trips (1 percent) and a larger increase in VMT (4.5 percent). A comparison of these model-induced travel results to the estimated actual induced travel results indicated that the model may underestimate induced travel effects somewhat for vehicle trips, VMT, and vehicle travel speed, and overestimate the reduction in travel speed. In addition, the land use results indicated that the expansion of both modeled and estimated actual induced travel tended to reduce employment in more established centers of the region and increased employment and household activity in the outer ring of the region. However, the model tends to overestimate the number of zones with smaller changes and underestimate the number of zones with larger changes (ranging from 1 to 19 percent) when model and actual induced travel estimates were compared.
Table 2 Summary of Tests Used in the Validation Studies
Integrated Land Use & Travel
Model Type Travel Demand (SACMET) (Sacramento MEPLAN)
Location Sacramento, CA Sacramento, CA
Time 9 years 10 years
Model Year 1991 2000
Total Model Forecast 2000 projected zonal
(demographic zonal & network demographics & network; NAinputs, functional forms, & MALPE1 of forecast relative to parameters) observed 2000 travel.
Forecast 2000 with 1990 observed zonal demographics,
Forecast 2000 observed zonal
1990 to 2000 observed growth Model Error demographics & network; MALPE rates, and 1990 to 2000 observed
(functional forms & parameters) of forecast relative to observed networks; MALPE of forecast 2000 travel.
relative to observed land use and travel.
Model Input (demographic zonal & network Total Model minus Model Error NA inputs)
1 MALPE is Mean Algebraic Percent Error.
2 Note that projection with 1990 or 1991 network is corrected for model error. NA is not available.
Table 3 Summary of Validation Study Results
Total Model MALPE1: (demographic zonal & +11.5 VMT network inputs, NA
functional forms, &
parameters) +38.4 VHD
1MALPE is Mean Algebraic Percent Error.2 MAPE is Mean Absolute Percent Error. 3 Res. is residential. NA is not available.
Table 3 Summary of Validation Study Results (Continued) Rodier (2004) Rodier (2006)
Model Input +5.8 VMT
(demographic zonal & +8.6 VHT NA network inputs)
Elasticity: Elasticity:
Estimated Actual & +0.22 (estimated -0.58 (estimated VMT/time) Model Induced VMT/Lane Miles) -0.46 (model VMT/time) Demand +0.14 (model +0.28 (estimated VMT/speed)
VMT/Lane Miles) +0.28 (model VMT/speed)
Special Notes NA Accuracy relatively good for central urban areas; positive bias in outer areas.
1MALPE is Mean Algebraic Percent Error.2 MAPE is Mean Absolute Percent Error. 3 Res. is residential. NA is not available.
Sensitivity tests are another approach to uncertainty analyses. These tests can measure the effects of uncertainty in one to all of the variables and parameters in a model on its forecasts. Numerous scenarios are typically simulated with randomly or nonrandomly generated values of separate variables or a combination of variables. In general, this approach has less intensive data requirements than validation studies. However, it can require many computationally time-consuming simulations (for example, 100+) and its relative conceptual complexity may make its results difficult to communicate. In addition, probabilistic methods require assumptions about the distribution of the errors in model input and parameters that may or may not be valid. Recently, several sensitivity analyses have been conducted on models used for transportation, land use, and/or air quality planning. 29
In this section, the results of univariate sensitivity analyses are described separately from multivariate sensitivity analysis. In univariate sensitivity analyses, a model is used to simulate scenarios in which one input variable varies over a range of plausible error levels to examine total uncertainty that may result from one input variable. In multivariate sensitivity analyses, a model is used to simulate scenarios in which more than one input variable is changed to unique, randomly, or nonrandomly generated values.
Demographic inputs to models, including projection of income, fuel prices, and the number and location of households and employment, are generally considered one of the greatest contributors to uncertainty in model forecasts. 30 Univariate sensitivity tests over a 20-year period were performed by Rodier and Johnston 31 on the 1996 Sacramento regional travel demand and emissions model, and by Rodier 32 on an early version of the Sacramento MEPLAN model and the region's emissions model. Plausible error ranges for income and fuel were obtained from available literature; error ranges for county-level population projections were developed by comparing past state population projections with subsequent performance. The results of these studies are presented in Table 4 . In general, the results suggest that errors in input population were a significant source of uncertainty in both the Sacramento travel demand model and the MEPLAN model. However, income and price were relatively more important sources of error in the Sacramento MEPLAN model than in the Sacramento travel demand model.
Table 4 Univariate Sensitivity Analyses
|Rodier & Johnston, (2002)
|Appendix A
|Travel (UTP)
|Integrated Land Use & Travel (MEPLAN)
|Sacramento, CA
|Sacramento, CA
|Time Horizon
|20 years
|20 years
-5.6 to 6.8 vehicle trips -30.2 to 42.5 vehicle trips -5.3 to 6.7 VMT Population -21.8 to 28.5 VMT -0.8 to 1.8 mean vehicle speed (±2 standard deviations or 95% -58.9 to 184.6 VHD
confidence interval ) -5.3 to 6.8 NOx -20.0 to 26.2 NOx -1.0 to 1.5 land consumption
0 to 9.7 vehicle trips
0.0 to 0.1 vehicle trips
0 to 3.1 VMT
Income 0.0 to 0.6 VMT
-1.3 to 0 mean vehicle speed
(extreme values) 0.1 to 2.9 VHD
0 to 3.1 NOx
0.1 to 0.4 NOx
0 to 3.3 land consumption
-0.8 to -0.4 vehicle trips
0.0 to 0.0 vehicle trips
-3.3 to 6.1 VMT
Fuel Price -0.2 to 0.2 VMT
1.9 to 2.1 mean vehicle speed
(extreme values)-1.4 to 0.7 VHD
-6.9 to 8.1 NOx
-0.1 to 0.1 NOx
0 land consumption
Population important but not All variables significantly contribute to Sources
income & fuel price. variability.
Model lacks sensitivity to cost Land activity & consumption outputs at
Notes zonal level more variable than travel variables.
A series of multivariate sensitivity tests have examined uncertainty due to input and parameter error on the forecasts of a number of travel and land use models used in planning studies.
In these studies, the models were run numerous times with unique, randomly, or nonrandomly generated sets of model input and parameter values. The studies and their results are summarized in
Table 5
Table 5 Summary of Multivariant Sensitivity Analyses
Zhao & Kockelman (2002)
Model type Travel Demand
Location Dallas/Fort Worth, TX
Time horizon Not reported
Demographic input &
Output variation1 ± 23.6 VMT (percentage change ± 23.2 VHT for 1 standard
deviation or 68% ± 189.9 mean congested
confidence interval) link time
Demographic inputs & Relative importance of parameters of trip sources of error generation most
1As measured by the coefficient of variation.
Krishnamurthy & Kockelmann (2003)
Land Use & Travel (DRAM-EMPAL)
Austin, TX
20 years
Monte Carlo
Demographic input & parameters
± 50 residential
density ± 37 employment
density ± 38 peak VMT ± 45 peak period
Most significant are exponent of link performance, peak & off-peak trip split, & trip generation & attraction rates; population growth rates less significant.
Pradhan & Kockelman (2002)
Land Use & Travel
Springfield, OR
15 years
Factorized Design
(variant of Monte
population &
rates, household
& mobility rates;
location choice
coefficients; land
price coefficient
± 7 land prices
± 6 occupancy
± 1 occupancy
rate ± 1 VHT ± 1 VMT ± 2 average flow
Population growth rates have significant long-run effect, but not mobility or accessibility.
Clay & Johnston (2005)
Integrated Land Use and Travel Model (MEPLAN)
Sacramento MEPLAN (version 2000)
20 years
Nonrandom Design
Selected: exogenous production; commercial trip generation rates; travel cost; concentration parameter
Not reported.
Commercial trip generation rates most important; followed by exogenous production, then concentration parameter, then travel cost.
The effects of input and parameter error in a smaller subset of a typical four-step travel demand model (see "Sensitivity Analyses of Models' Theoretical Validity " on page 17 for more detailed descriptions of the four-step model), the Dallas/Fort Worth Model, was examined by Zhao and Kockelman. 34 Model simulations were made for each of the 100 unique sets of input and parameter values generated by the Monte Carlo random sampling method from the complete set of model inputs and parameters. The results indicated that the input errors tended to propagate through the first three model steps of trip generation, trip distribution, and mode split. However, at the final step, trip assignment, errors were reduced but not below the initial input error levels. Sensitivity tests were also performed, to test the relative significance of different sources of input and parameter errors on model output by regressing outputs on the input variables. Household and employment inputs to and key parameters of the trip-generation step were found to be significant contributors to output variation in VMT and VHT. The total uncertainty of the model (measured as the coefficient of variation) due to input and parameter errors suggested that there is a 32 percent chance that the model will over- or underestimate VMT and VHT by approximately 23 percent (1 standard deviation or 68 percent confidence interval) and only a 5 percent chance that it will do so by approximately 46 percent (2 standard deviations or a 95 percent confidence interval). 35
A similar uncertainty analysis was conducted by Krishnamurthy and Kockelman in 2003, but in that study the subject was the Austin-calibrated DRAM-EMPAL model linked to the regional travel demand model. 36 Using a Monte Carlo sampling approach, 200 random sets of model parameters and inputs were drawn from the complete model set (95 parameters and population and employment growth rates) and 200 simulations were conducted for a 20-year time horizon. The most significant sources of variation were found to be "the exponent of the link performance function, the split of trips between peak and off-peak, and several trip generation and attraction rates." 37 Errors in employment and population growth rates were found to be significant in the long run but not the short run. The analysis of total model uncertainty resulting from input and parameter errors (measured as the coefficient of variation) suggests that for a 20-year period there is a 32 percent chance that the model will overestimate or underestimate residential density by 50 percent, employment density by 37 percent, and peak VMT by 38 percent (1 standard deviation or 68 percent confidence interval). 38 Thus, there is only a 5 percent chance that the model will overestimate or underestimate residential density by 100 percent, employment density by 74 percent, and VMT by 76 percent (2 standard deviations or a 95 percent confidence interval).
Pradhan and Kockeman investigate the uncertainty in yet another land use and travel model, the Eugene–Springfield, Oregon, UrbanSim model. 39 This study uses a variant of the Monte Carlo sampling method, the factorized design approach, to efficiently select a range of well-distributed values (81) for selected model inputs and parameters, including population and employment growth rates, household and mobility rates, location choice coefficient, and land price coefficients. Eighty-one unique sets were selected and simulated over a 15-year period. Again, the sensitivities of outputs from the land use and travel demand model were determined by regressing key outputs on the selected input rates and coefficients. Among the selected inputs evaluated in the study, only population and employment growth rates were found to have significant long-run effects on output variation. The results of the study also indicated that land prices and occupancy density outputs were significantly more variable than travel output (as measured by the coefficient of correlation); 1 standard deviation or 68 percent confidence interval was approximately ±1 for VHT, VMT, and the occupancy rate compared to approximately ±6 and ±7, respectively, for land prices and occupancy density. 40 These output variations reflect the uncertainty due to the limited variables selected for analysis.
Clay and Johnston's 41 examination of uncertainty in the Sacramento MEPLAN model (calibrated to year 2000 data) is similar in approach to Pradhan and Kockelman. 42 However, this study uses a nonrandom design to generate 239 unique sets of selected inputs (exogeneous production, commercial trip generation rates, cash costs of driving a single-occupant vehicle [SOV], and concentration parameters) from three increasing incremental levels of variation for each input. Like the previous studies, the sensitivities of outputs from the land use and travel demand model were determined by regressing outputs on the selected input variables. The results indicated that all inputs were significant, but commercial trip generation rates were most significant, followed by exogenous production, then the concentration parameter, and finally the cash costs of driving an SOV. Unlike the Kockelman studies, this study did not report the total variation in output values.
Another group of recent sensitivity analyses in the United States have been conducted to investigate the bias in model forecasts when the basic economic relationships upon which the models were originally based are not adequately represented—more specifically, how a change in transportation system supply changes the time and monetary cost of travel, and thus the total travel demand, all else being equal. A relatively large body of literature has provided empirical evidence for this relationship (also known as induced demand or travel). For example, in the near term (a few years), the travel effects from new highway capacity include changes in destination choice (or trip distribution); in the longer term (more than 10 years), effects can include changes in households and employment location, development, and land consumption. 43 The empirical literature indicates that the elasticity of VMT with respect to lane miles (the most common measure of induced travel) ranges from 0.3 to 1.0 in the long term. 44 The magnitude of the bias due to models' failure to adequately represent the supply-and-demand relationship is important because of its implications for fair analyses of competing transportation scenarios (for example, highway, transit, and no-build).
It is important to note that the studies reviewed in this section do not assert that these models accurately represent the specific behavioral components of induced travel or that the theory upon which these models was originally based is valid. The objective is to evaluate the capability of current travel demand models to represent induced travel (relative to the existing empirical literature) and the analytical consequences of not representing induced travel in transportation planning and policy studies. Currently, most travel demand models used in the United States contain some sort of bias related to the representation of induced travel effects. As outlined in "What the Experts Have to Say: Improving and Applying Uncertain Models in Transportation Planning" on page 21 , progress in developing the next generation of models has been extremely slow. Therefore, it is important that stakeholders understand what improvements can be made to existing models and how these improvements can address their concerns about the effects of new transportation projects.
To date, sensitivity analyses of induced demand effects have been conducted on models developed for the Sacramento, Chittenden (VT), and Salt Lake City (UT) regions in the United States. 45 These models include a number of induced demand effects (including land use, trip generation, trip distribution, mode choice, and traffic assignment) and consistently represent input and output travel time and cost values (the time to travel from point A to B by mode C is 5 minutes in every step of the model). The total and relative contributions of these effects are explored by turning on and off model components that represent different components of induced travel effects (see Table 6 ).
Table 6 Model Components and Variables in the Case Study Models
|Model Components
|Sacramento (CA)
Chittenden (VT)
Salt Lake City (UT)
|Land Development (acres of land developed)
|Modal travel time & cost
|Activity Allocation (where urban activities locate)
|Modal travel time & cost
|Modal travel time & cost
|Modal travel time
|Trip Distribution
|Modal travel time &
|& cost for work
|Modal travel
|Auto travel
|(origin and destination of trip)
|trips; auto times for others
|time & cost
|time & cost
|Mode Choice
|Modal travel time &
|Modal travel time
|Modal travel
|Modal travel
|(mode use in trip)
|& costs
|time & cost
|time & costs
Traffic Assignment Modal travel time & Modal travel Auto travel
Auto travel times
(route/road taken for trip) cost time & cost times Dashed areas indicate absence of model components.
In the Sacramento region, tests were conducted on the integrated land use and transportation model, the Sacramento MEPLAN model (calibrated to 1990 data), 46 and the SACMET (regional travel demand) model (1996 version). 47 In the Chittenden case study, tests were conducted on their regional travel demand model linked to a land allocation model. 48 In the Salt Lake City study, tests were conducted on their regional travel demand model. 49 These models all iterate or "feed back" modal travel times and/or costs among their submodels until convergence values (or consistent model input and output of travel time and/or cost) are achieved. All the case study models were official metropolitan planning organization (MPO) models, with the exception of the Sacramento MEPLAN model, which was the earlier version of the model that was subsequently updated and adopted for use by the regional MPO.
Sensitivity tests were developed to assess the contribution of each model step to the model's total representation of induced travel in the network scenarios. Again, this is accomplished by turning on and off different model components or steps. An illustration of the sensitivity tests is provided in Table 7 . The first sensitivity test, A, is simulated with the full model to represent all induced travel effects. Each subsequent sensitivity test, B to D, drops an additional submodel component by holding it constant from the no-build scenario. For example, sensitivity test B holds land uses constant from the no-build scenario and simulates only the trip distribution, mode choice, and traffic assignment effects of a transportation scenario.
Table 7 An Example of Sensitivity Tests
Land Use – – No-build land uses No-build land uses No-build land uses
Trip Distribution –– –– No-build trip tables No-build trip tables
No-build mode Mode Choice –– –– –– choice
Dasehd areas indicate model components held constant from the no-build.
The results of the simulation tests with the Sacramento integrated land use and transportation (MEPLAN) model 50 indicated that change in land use patterns from the new highway capacity over a 20-year time horizon accounted for half the predicted induced travel, and the change in trip origin-destination patterns (or trip distribution component) accounted for the other half. Overall, the model's long-term representation of induced travel (elasticity of VMT with respect to lane miles) for new highway projects was 0.8. This figure is consistent with the high end of the empirical range in the literature (as described above). The percentage underestimation of the travel and emission effects from the highway to the no-build with and without full model feedback would be 102 percent for VMT and 192 percent for NOx (oxides of nitrogen) emissions (see Table 8 ).
Table 8 Long-Term Induced Travel Sensitivity Test Results with the
Case Study Models
0.23 (lane miles) 0.76 (lane miles)
Elasticity of VMT 0.8 (lane miles) 0.78 (lane miles)
- 0.41 (travel time) - 0.66 (travel time)
Submodel Elasticity Contribution
Land Development 25% (lane miles) – – – – – – -1% (lane miles)––
Activity Allocation 25% (lane miles) ––
2% (travel time) 113% (lane miles) 71% (lane miles)
Trip Distribution 50% (lane miles) 53% (lane miles)
112% (travel time) 76% (travel time)
Dashed areas indicate model components held constant from the no-build
Table 8 Long-Term Induced Travel Sensitivity Test Results with the
Case Study Models (Continued)
HIGHWAY Sacramento (CA)
-9% (lane miles)32% (lane miles)
Traffic Assignment 0% (lane miles) 47% (lane miles)
5% (travel time) 23% (travel time)
Percentage Underestimate: No Feedback VMT 102% 94% 70% 85% VHT 192% 16% 236% –– NOx Emissions 25% (lane miles) 72% – – – – Dashed areas indicate model components held constant from the no-build
Similar simulation tests were conducted with the region's travel demand model (SACMET), which does not include a land use component. 51 These results indicated that, for a 20-year time horizon, the model predicted an elasticity of VMT with respect to lane miles of 0.23 and an elasticity of VMT with respect to travel time of -0.41. These figures are consistent with the very low end of the empirical elasticity range described above. The sensitivity tests indicated that the change in origin-destination trip patterns from the highway projects (enabled by full feedback to trip distribution) accounted for almost all of the model's representation of induced travel. The negative results for mode choice and traffic assignment suggest that this model would forecast a reduction in VMT relative to the no-build without full feedback. The percentage underestimation of the travel and emission effects from the highway to the no-build with and without full model feedback would be 94 percent for VMT, 16 percent for VHT, and 192 percent for NOx emissions.
The results of the Chittendon case study 52 indicate that the trip distribution component accounted for almost 75 percent of the model's representation of induced travel, and the traffic assignment component accounted for almost 25 percent. The elasticity of VMT with respect to lane miles was 0.76 and with respect to travel time was -0.66. The land use effect in this scenario was negligible. Over the 25-year time horizon, additional roadway miles were forecast to be only about one-tenth of the growth in households and employment. As a result, the congestion effect (due to population growth) on the networks tended to swamp any increase in capacity. Even without significant land use effects, the percentage underestimation of the travel effects from the highway to the no-build with and without full model feedback would be 70 percent for VMT and 236 percent for VHT.
The Salt Lake City case study
indicates that the changes in trip distribution and traffic assignment from the new highway project each accounted for about 50 percent of the model's prediction of induced travel. The elasticity of VMT with respect to lane miles for the highway alternative was 0.78. The percentage underestimation of the travel from the highway to the no-build with and without full model feedback would be 85 percent for VMT.
A series of interviews was conducted with modeling experts in the field of transportation and environmental planning to gain insights into the failures and successes of modeling, the factors driving improvements and problems, and recommendations on steps that might be taken to accelerate the advancement of the state of the practice. This section begins with a description of the expert interview process. Next, the results of the expert interviews are presented. The section concludes with a number of recommendations to address the institutional barriers to the advancement of models used in transportation planning.
Interviews were conducted during January, February, and March of 2006 with modeling experts having experience in federal, state, and regional government agencies, as well as academia, nongovernmental organizations, and private consulting. The policy focus of these experts included both transportation and environmental planning. Ten in-depth expert interviews were conducted. An interview guide was developed that probed experts about the state of the practice and state of the art in travel demand models, institutional factors encouraging and discouraging improvements, and recommendations to improve the state of modeling (see Appendix B ). For the protection of the experts, they were assured that their identities would be kept confidential.
When asked about the state of modeling practice, the experts interviewed for this study almost unanimously characterized it as poor. The experts used words such as "dismal," "primitive," "disappointing," and "deficient" to describe the state of the practice.
Several experts pointed out that the state of the practice varies by the size of the Metropolitan Planning Organization (MPO). In large MPOs (population of one million or more), some improvements have been made to the traditional four-step models. A few of these MPOs are experimenting with advanced activity, tour, and land use models. However, another expert noted that many large MPOs make defensive improvements to protect themselves from lawsuits related to the need and environmental effects of new roadway projects.
In medium MPOs (more than 250,000 and less than 1 million) and small MPOs (less than 250,000), the practice of modeling is poor. One expert asserted that small MPOs might not need more sophisticated modeling because their transportation problems may be relatively simple and their range of alternatives may not include transit. Another, however, suggested that the lack of modeling sophistication in small MPOs may preclude the analysis of transit alternatives to roadway expansion. The experts explained that the larger MPOs tend to have more resources than smaller MPOs to support higher levels of modeling practice. Another expert stated that many smaller MPOs have their state department of transportation (DOT) do the modeling, but would likely do it themselves if they had the resources.
The experts were also asked to describe advances in the state of modeling practice. Several commented that there has been a general shift from using zones as the unit of analysis in models to using travelers, thus reducing aggregation error. Several noted the development of more sophisticated assignment methods (for example, equilibrium assignment) that simulate the route-diverting effects of congestion on a particular route. Some commented that models are more commonly operated with feedback of travel times (from the final model step to earlier model steps) to achieve consistent travel times and provide some representation of induced travel. However, two experts asserted that feedback is neither correctly nor consistently implemented in many of these models. Experts also reported that some four-step travel demand models have been modified to better represent roadway alternatives including, for example, transit investment, nonmotorized modes, and transit-oriented developments. However, many asserted that these modifications are not sufficient to allow a fair evaluation of alternatives. Finally, some MPOs have linked their travel models to land use models. One expert noted that almost all these advances were introduced in the 1970s and 1980s.
In general, the experts expressed frustration that even the more advanced state-of-the-practice models are unable to adequately simulate the effects of transportation investments, land use measures, and pricing policies for two reasons. First, models lack the variables to sufficiently represent the quality (for example, spatial resolution, time, and cost) of alternatives to highway investments. Second, even if the models could represent these supply variables, they are unable to adequately show how changes in these variables influence individuals' location, destination, mode, and departure time choice.
Many experts noted that operating a model with feedback to trip distribution (and variable trip tables) can show how alternatives, assuming they are adequately represented in the model, influence the traveler's destination choice, such as where to shop and work. As indicated above, this process is more common today, but two experts stated that many MPOs implement the process incorrectly and inconsistently. Another expert stated that split or windowed models, which are frequently used in environmental impact analyses of new roadway projects, typically do not adequately represent feedback to trip distribution. In addition, smaller metropolitan areas commonly use fixed trip tables when evaluating new projects.
The experts also noted the importance of the integration (or linking) of a land use model with a travel demand model to show how transportation alternatives (again, if adequately represented) affect residential and employment location choice and regional development patterns. Many stated that travel demand models depend on accurate land use inputs; however, many land use models are inadequate, and many regions use the same land use projections in build and no-build scenarios. For example, one expert stated that because there is no representation of land use and transportation interactions, future land use forecasts are not credible, so forecasts of long-run transportation plans are also not credible. Another expert stated that tools are needed that can simulate the land use and transportation interaction and show how people adapt to the built environment, for example, whether demand will go away if a no-build decision is pursued because people make different decisions about where they live, shop, and work.
The inability to represent departure time choice or peak spreading in models was raised by most of the experts interviewed. Many models, it was noted, are still models of daily traffic, but even if a model does represent multiple time periods (for example, A.M. peak, off-peak,
P.M. peak), they are fixed and based on relatively arbitrary factors. Many stated that the failure to represent peak spreading under congested conditions may result in an overestimation of congestion and the demand for new roadways. The ability to represent peak spreading is also critical to analyzing time-of-day road pricing. Others noted that correct speeds (including accelerations and decelerations) are critical to air quality analyses.
Another consistent theme in the expert interviews was that the models used in transportation planning were originally developed (as far back as the 1950s) to evaluate alternate capital investments in large-scale highway facilities. Now these models, with minimal improvements, are being asked to evaluate alternatives that they were not designed to evaluate, for example, pricing policies, transit-oriented development, transit, and walk and bike facilities. Several experts noted that it is almost impossible to adequately evaluate such policies with current models, so there is a systematic bias for transportation investment alternatives and possibly against more cost-effective demand management strategies. Experts pointed to some specific concerns, including poor model spatial resolution (zones); inability to model walk and bike trips, which typically occur within rather than between zones; lack of pricing variables; and inadequate representation of goods movement. One expert commented that over the next 15 years, the greatest source of most of the growth in air pollutants will be goods movement, and the current four-step models are badly suited to this task.
Given the shortcomings of current models, the experts were asked to share their thoughts about whether and how modeling uncertainties should be communicated. The responses were varied. One expert expressed concern that policymakers do not have the technical background to understand the expression of modeling uncertainty. Another cited one region as an example of an MPO that clearly communicated more complicated modeling analyses. One expert stated that it was too costly for MPOs to conduct uncertainty analyses, given current funding levels. Many experts felt strongly that the uncertainty in any particular modeling analysis should be made as explicit as is possible, and that the presentation of results as point estimates, rather than with confidence intervals, was misleading. One specifically stated that the presentation of such results would help the public understand that the outputs from these models are not certain. Two noted that the current conformity process does not allow for the expression of uncertainty and makes it difficult to use models that produced a range of outcomes within some confidence interval (for example, TRANSIMS). One asserted that environmental impact statements, under the National Environmental Policy Act (NEPA), did allow for uncertainty analyses. Many experts, however, agreed that modeling a range of alternative or strategic planning scenarios to identify which investments and policies would perform best, under different and uncertain conditions, was a good way to deal with modeling uncertainty. They also indicated that the use of microsimulation tools to model travel behavior and land use decisions would facilitate such analyses.
The discussion of communicating the uncertainty of model results raised questions about the state of ethics in the practice of modeling. One expert explicitly stated that there is a very low level of ethics in the field with respect to self-reporting on modeling problems and shortcomings. Moreover, there are no resources available to provide the oversight necessary to verify modeling quality. Almost all the experts indicated that models too frequently are used principally to justify decisions that have already been made, rather than to help inform the decision-making process. One stated that the majority of MPOs in the country are just going through the motions: if the model gives answers they do not like, rather than disclosing the results, they tweak the model to get the results they want, for example, to show conformity and secure transportation dollars. On the other hand, advocacy groups may send in modeling consultants who manipulate results to show that a project is not really needed and will cause the region to exceed their emissions budget. As one expert asserted, there are serious ethical challenges facing the field of transportation modeling.
The experts interviewed for this study generally agreed that activity- and tour-based modeling using microsimulation and integrated with land use models is the state of the art. Together, they stated, these models allow movement from the aggregate four-step travel demand model by abandoning the zonal system, using synthetic populations, and geo-coding individual households. It was also noted that these models move from treating demand for a good as fixed to demand that is responsive to supply. One expert emphasized that land use models are also moving to simulation of parcel-level residential and employment choice. In general, they stated that the focus of these models is on a more explicit representation of decision and choice processes rather than just system performance. They emphasized that activity models represent the effects of household constraints (for example, child-care responsibility) that often influence mode and route choice; it is not possible to represent such effects in trip-based models. Many pointed out that advanced traffic simulations allow for the representation of the effects of congestion, queues at bottlenecks, signal timing, left turn lanes, time of day/peak spreading, and road pricing policies. They noted that this is important because these new models support the new operation and maintenance trend in transportation: less building and greater management of existing facilities.
The speed at which the state of the art of modeling is advancing was of some debate among the experts. One asserted that it is advancing far more slowly that in Europe. Another asserted that it was advancing fairly rapidly. The latter expert also stated that a half dozen MPOs are developing these tour- and activity-based models; over the next couple of years, activity-based modeling linked to microsimulation and disaggregate microsimulation should become widespread.
Experts identified areas in the United States that have implemented or are planning to implement elements of state-of-the-art models. For activity- and/or tour-based models, these areas include New York, Portland, Sacramento, Atlanta, Denver, Baltimore, and Columbus. For integrated land use and transportation models, these include Utah, Hawaii, Texas, Oregon, Ohio, Sacramento, Portland, and San Diego.
Two experts expressed some reservations about the practical applications of state-of-the-art models. One asserted that activity-based models are good in the long term, but current models have many weaknesses that could be addressed sooner and more cheaply. He stated that the models in New York, which take three to four days to run, are considered advanced but do not simulate land use effects. Another stated that his MPO, which is developing an activity-based model, will run their four-step model along with their new activity-based model for some time.
Some experts pointed to specific models: TRANSIMS, PECAS, and UrbanSim. TRANSIMS is a computationally complex system with large computing capacity that includes microsimulation of trips and the potential to simulate goods movement. This tool is not used in practice, but there have been tests in Portland and Dallas/Fort Worth, and the effort has spurred some commercial development. PECAS, an integrated land use and transportation model, and UrbanSim, a land use model that can be linked to a travel model, represent the economics of developer actions and location choice. One expert stated that the potential benefits of these models include more accurate forecasts of demand for passenger and goods movement; analysis of interregional transportation and land use projects and alternatives; and more sophisticated analysis of alternative performance, including cost and benefits, equity, and environmental.
Factors Driving Progress
The experts were asked what they thought were the most important factors driving modeling improvements. Many experts noted that increasingly, at the local and regional level, there are more questions from the public about the effects of new highways and transit on the growth of their communities. One pointed out that many stakeholder groups, such as the Environmental Defense Fund and Sierra Club, have grown more technically sophisticated and are asking specific questions about how the model represents induced travel, land use effects, and alternatives to highways. He further stated that the failure to adequately address stakeholders' concerns about the adequacy of the models used to evaluate alternatives has resulted in costly lawsuits in a number of regions. The experts generally agreed that more advanced MPOs have attempted to develop models that can begin to address the public's questions and concerns.
The experts all agreed that the 1990 Clean Air Act Amendments and subsequent Air Quality Conformity Requirements are major factors driving model improvements. These requirements, they stated, raised the bar on model performance for emissions testing, and the conformity rule included specific language about the operation of the model (for example, full feedback in nonattainment areas). One expert indicated that the 1991 Intermodal Surface Transportation Efficiency Act (ISTEA) also played an important role. Other experts cited the cumulative impact assessments and air quality analyses required by NEPA. One expert cited the Environmental Justice Presidential Executive Order, which requires agencies to evaluate the effects of their actions on minority and poor households; thus models must be able to assess the effects of plans and programs on households by income class, location, and, if possible, by minority status.
Almost all the experts agreed that advances in computing and information technology have significantly aided model development efforts. It was stated that computing and technology improvements allow much more computationally complex models (for example, activity and land use models) to run on personal computers. They also indicated that it is easier and cheaper for models to be supported by good global information system (GIS) data and to integrate that with data sets. Another expert emphasized the importance of the greater accessibility to less-expensive data needed for model development.
As described above, most experts stated that the changing emphasis in transportation planning from major capital investments to demand and operational management of existing capacity is another major factor driving model improvement. Current four-step modeling tools are not capable of adequately simulating toll roads, pricing policies, and traffic operations. One expert stated that as we move forward in the next decade, a key driver will be the move toward toll financing, use of tolls as traffic and operations management, and the use of public-private partnerships to finance and manage transportation infrastructure.
Factors Driving Stagnation
All the experts indicated that the most important barriers to model improvement efforts have to do with the system of transportation financing and its bias in favor of capital investment projects. One expert stated that models are not used to analyze alternatives, they are used to justify projects that powerful stakeholders want to advance their own interests or objectives (for example, developers' need for transportation projects to support new development projects and politicians' desire for ribbon-cutting ceremonies and federal money for local jobs). In addition, many agencies and their staff have a conflict of interest because their institutional survival depends on new roadway projects.
Many experts agreed that the lack of independence between the agencies doing the modeling and the principal sponsors of capital projects drives modeling problems. One stated that an MPO's primary task is to get federal dollars for transportation projects, so MPO managers do not see models as useful unless they can bring in money. Another noted that it is hard to get politicians to look at the results of more complex models (such as land use and transportation) that take longer to operate. Another expert indicated that qualified modeling staff can often be under pressure from their management to come up with certain political outcomes. However, one noted that some MPOs use models to examine public choice in a reflective way, for example, Portland, Sacramento, and Albany. This expert asserted that the modeling conducted for Toronto (Canada) is excellent because it is operated at the University of Toronto, which is independent of the regional government.
One expert expressed concern that a disproportionate amount of money is put into capital infrastructure relative to the amount that is put into the planning of that infrastructure. All experts agreed that more resources need to be allocated for model development and improved data. More money is needed for data acquisition, management, and upkeep. The new models need more high-quality and detailed data on, for example, household and developer behavior and land use. The experts also mentioned that the lack of oversight and independent auditing of modeling is a significant barrier to progress. One expert stated that the typical long-range plan and project analysis is 20 to 30 years in the future, which is typically well beyond the average career time of most planners. As a result, no one ever goes back and checks the accuracy of the forecasts. Overall, he said, it is difficult to retrospectively see how well the cost and outcome of the project was predicted. However, he believed an increased emphasis on public-private partnership for highway projects (construction revenue through tolls) might encourage greater accountability. Another expert indicated that a rigorous economic analysis of project viability might improve the process. Earmarking was unanimously criticized by the experts as a disaster for modeling.
All the experts interviewed for this study indicated that the gap between the state of the art and the state of the practice is widening, in part because fewer and fewer technically skilled modelers are going into the public sector, in particular MPOs. In addition, many MPOs, because of funding limits and regulatory requirements, do not have the resources to hire planners who are dedicated to modeling, and thus must hire generalists who can, for example, also conduct public meetings and help with the long-range plan. As a result, those who understand and administer the models seldom are capable of increasing the complexity of modeling. Moreover, if an MPO does have qualified staff, they are difficult to retain because existing salary structures are so low. As a result, such staff often is hired away by consulting firms or software companies that can pay a higher rate. Model oversight by the Environmental Protection Agency (EPA), the Army Corps of Engineers, and state DOTs is weak, in part because they typically do not have any or enough staff that is qualified to review modeling analyses.
The experts interviewed for this study had several recommendations that may help move the practice of modeling forward in the United States. These tended to relate to greater investment of public dollars or more effective use of public dollars to provide incentives for model improvement (the carrot) and to develop effective processes of auditing and oversight for modeling (the stick).
Given the preceding discussions, the most obvious recommendation was to invest more public dollars to improve model development and application efforts. For example, funds could be awarded to MPOs that demonstrated commitment to improving modeling practices and linking modeling analyses to decision making. This could include long-range planning that involves the public in regional visioning scenario analyses, continuing education for agency modelers, and documented comparisons of the results of activity-based models and four-step models. However, as one expert stated, a general increase in funding for modeling is not an easy case to make at the present. There is some hope that profits from the private sector (consulting firms) may be able to move the state of the practice closer to the state of the art.
The experts had many ideas on how to implement independent auditing and oversight of modeling. The general consensus was that this should involve the development of explicit modeling standards and independent and routine auditing process to certify a model. Several experts favored an approach that included tighter modeling regulations, requirements, or guidance documents with periodic evaluations. Moreover, MPOs with air quality conformity problems would be required to have their model certified each year. However, another expert suggested that a voluntary star rating system, which rewards agency models that achieve a high level of excellence, may be easier to implement. Another expert recommended that agencies be required to collect data for retrospective checks on how well the project outcome was predicted. Several mentioned that the current peer review process for modeling is not effective because EPA and DOT do not have the institutional capacity to fairly evaluate models. Another noted, however, that DOT has made some progress toward incorporating some elements of model review in the MPO certification process. Several also suggested that a consortium of independent universities should develop modeling standards or conduct modeling for agencies.
In this section, case studies are used to illustrate innovative modeling approaches and future directions. A wide range of approaches are reviewed because the choice of one approach over another will be dictated by the objectives of the analysis, the modeling tools available, and the knowledge of uncertainty in the modeling tools.
Current regulations require U.S. transportation planning agencies to provide point estimates of travel and environmental effects for a single transportation plan or for comparisons of a transportation plan to a do-nothing or no-build scenario. For example, regions in air quality nonattainment areas must demonstrate that future emissions forecasts for their transportation plan are within (or "in conformity" with) their allowed emissions' budget. In addition, environmental impact statements must include and compare forecasts of travel and emissions for the proposed project to a do-nothing scenario. In this section, the results of the many uncertainty analyses described in "Synthesis of the Literature on Model Error and Uncertainty" beginning on page 29 are applied in the following case studies to illustrate how such analyses can improve the credibility of forecast required by current regulations. Analyses of error and uncertainty may also improve the general policy process by making the users of model results aware of the model's uncertainty. As a result, the focus of the analysis may shift from meeting a point estimate of demand for travel in a particular corridor and toward the rank ordering of a number of alternative policy strategies. It may be far more defensible to use an uncertain model to compare competing alternatives rather than projecting and meeting a particular point estimate, as long as the model's structure is not biased toward particular modes or policies. The evaluation of a range of alternatives is more likely to address stakeholder concerns and encourage innovative thinking about the future.
Case Study A: Univariate Sensitivity Analysis for Conformity. As described previously, a sensitivity analysis of plausible errors in population and employment was conducted using the travel demand and emissions models of the Sacramento, California, region for their transportation plan. 54 The results of the analyses indicated that plausible errors in population and employment projections (within approximately 1 standard deviation) may result in the region's transportation plan not meeting the conformity test for NOx in the year 2005 (an approximately 16 percent probability). This outcome is also possible in the year 2015 but less likely (within approximately 2 standard deviations or a 2.5 percent probability).
Case Study B: Tests of Model Accuracy for Conformity and Environmental Impact Analyses. Validation tests of model accuracy, as described previously, were performed on both the Sacramento region's travel demand model 55 and Sacramento MEPLAN over approximately 10 years. 56 The results of the tests of model accuracy for a SACMET travel demand model show that the model (excluding input forecast errors) overestimates VMT by 5.1 percent, VHT by 4.2 percent, and VHD by 17.1 percent. If the model were used for conformity analyses, its overestimation of daily vehicle travel would provide a relatively generous margin of error with respect to meeting air quality emissions budgets (again, assuming the magnitude of change for VMT forecasts is consistent with those for NOx). However, in the analysis of travel effects of proposed highway investment projections in environmental impact statements, overestimating the daily travel results would tend to overestimate no-build travel demand and congestion, and thus the need for new highway projects in the region. Compared to the no-build alternative, the magnitude of change for the highway alternative would have to be greater than the model error to be considered significantly different, which may be a difficult standard for the typical new highway project to meet.
The results of the tests of model accuracy for a Sacramento MEPLAN model travel demand model suggest that if the model were used in conformity analyses, then the regional transportation plan emissions analysis should fall outside the 3 percent model error underestimate to demonstrate conformity (again, assuming the magnitude of change for VMT forecasts is consistent with those for NOx). If the model were used to analyze the travel effects of proposed highway investment projections in environmental impact statements, the overestimation of daily travel results would tend to overestimate no-build travel demand and congestion, and thus overestimate the need for new highway projects in the region. Compared to point estimates for the no-build alternative, the magnitude of change for the highway alternative should be greater than the absolute value of model error to be considered a significant improvement over the no-build alternative.
Over the past 10 years, regional "visioning" analyses have become increasingly important. Visioning refers to scenarios that allow stakeholders to explore how their community goals can be achieved through alternative futures, rather than just one point-estimate of the future. Such a visioning exercise may include a no-build scenario, a highway-oriented scenario, a transit-oriented scenario, a pricing scenario, or some combination of elements of alternative scenarios. The following is a list of some of the more well-known regional visioning analyses:
The community goals of many visioning activities are exemplified by the Metropolis Plan in Chicago, as described by Marshall and Grady:
We can build a better region. We can spend less time in traffic. We can live nearer to our jobs. We can protect more open space and environmentally sensitive areas. We can build communities that are friendlier to walking and biking—and therefore healthier for the people who live in them. We can make economic opportunity available to more of our region's residents. 58
As discussed previously, there are many limitations to the current four-step travel demand models, so these regional visioning scenarios almost always require model improvements to increase their sensitivity to policy alternatives of interest to stakeholders. As Marshall and Grady 59 state, this frequently involves:
Marshall and Grady 60 predict that
Regional visioning and scenario analysis studies will be increasingly popular in the United States as regions seek alternatives to conventional long-range transportation planning that better meets citizens' needs. Travel demand modeling is an essential component of these analyses. In the near term, most of this work will be done with four-step models. Practical enhancement can be made to four-step models that will make the modeling as realistic and useful as possible.
The results of error analyses can be used to specify evaluation criteria for a wide range of future transportation and planning alternatives or "visioning" scenarios. Stakeholders in the regional planning process commonly complain that there are only small differences among scenarios. Such analyses can be used to place scenario differences into perspective and to alert stakeholders to the magnitude of investment or policy change required to achieve more certain policy goals, reduction in air pollutants, and roadway congestion, as suggested by the following case study.
Case Study C: "Significant" Scenario Results. The results of the univariate sensitivity analyses conducted with the Sacramento MEPLAN model, 61 described previously, can be applied to illustrate significant alternative scenarios, including transit and roadway investment, auto pricing policies, and land use measures. If the results of an alternative scenario do not fall outside the confidence intervals established in the sensitivity analysis, the results for the alternative scenario cannot be considered significantly different. The results of this study indicated that at the regional level the output variation in land consumed is relatively large. Land consumption results from transportation investment, and auto pricing scenarios in the alternative scenario analysis did not typically fall within the confidence intervals for population and income input variation; however, those scenarios that included the land use measures did. The level of variation produced from the population, income, and fuel price sensitivity scenarios for vehicle travel and emissions results were relatively moderate, and the alternative scenarios typically fell outside the 95 percent and extreme value confidence intervals. The exceptions are some of the more moderate auto pricing scenarios and the transportation investment scenarios (that is, the HOV and the LRT-only scenarios).
When a model cannot represent induced travel and this is at issue for a proposed new roadway project, one practical and low-cost solution is to have stakeholders identify the objective of the project (for example, level-of-service D for a roadway project in 20 years) and then adjust traffic inputs to establish the change in travel associated with the change in capacity that would result in the failure to realize the objective. If this change were outside of the upper range in the induced travel literature, then the project would have a high probability of meeting its objective with respect to uncertainty in induced travel.
Case Study D: Break-Even Modeling. Stathopoulos and Noland 62 conducted simulation studies that illustrate the approach outlined above:
Two scenarios for improving traffic flow are simulated and analyzed using the VISSIM microsimulation model and the Comprehensive Modal Emissions model. Short-run and long-run emissions of CO, HC, NOx, and CO2 and fuel consumption are estimated. In the short run, with traffic volume held constant, results demonstrate that the smoothing of traffic flow will result in reduced emissions. Long-run emissions are simulated by synthetically generating new trips into the simulated networks to represent potential induced travel effects. This is done until a "break-even" level of emissions for each pollutant and fuel consumption is reached that is equivalent to the base level before the traffic flow improvement was added. By also calculating short-run changes in travel time from the improvement, the travel time elasticity equivalents for each pollutant are calculated. These values are compared with travel time elasticities in the literature to evaluate whether long-run emissions benefits are likely to endure. Simulations are conducted using different assumptions of vehicle soak time to simulate cold-start and hot-stabilized operating modes. Results indicate that, in most cases, long-run emissions reductions are unlikely to be achieved under the two scenarios evaluated.
In a recent article, Popper, et al. 63 suggest an adaptive approach that more fully combines uncertainty analyses with visioning or heuristic modeling. They critique the visioning approach, as described below:
Although scenario analysis avoids making definite predictions, it has its own shortcoming. It addresses no more than a handful of the many plausible futures, so skeptics can always question the choice of the highlighted few. More fundamentally, scenario families do not translate easily into plans for action. How should decision makers use the scenarios? Should they focus on the most threatening case or the one regarded by experts as most likely? Each approach has faults.
They propose, instead, "to look for not optimal strategies but for robust ones."
These should exploit computational power as well as the best available knowledge and data to model "stress-test candidate strategies, searching for plausible scenarios that could defeat them."
These plausible strategies can take into consideration the relative uncertainty in the models' assumptions (for example, technological change, social change, and legislative factors), input data, parameters, and structures. This would be a more sophisticated application of Case Study C, described above. Not only will this process, as they assert, "reveal futures in which the proposed strategies could perform poorly," but "it also highlights ways each strategy could be adjusted to handle those stressful futures better."
In this study, an integrated land use and transportation model, the Sacramento MEPLAN, is used to simulate plausible errors in input population, income, and employment to set confidence intervals on forecasts. The results of error analyses are then used to specify evaluation criteria for a wide range of future transportation and planning alternatives or "visioning" scenarios. Stakeholders in the regional planning process commonly complain that there are only small differences among scenarios. Such analyses can be used to place scenario differences into perspective and to alert stakeholders to the magnitude of investment or policy change required to achieve more certain policy goals, reduction in air pollutants, and roadway congestion, as suggested by the following case study.
MEPLAN belongs to the family of integrated transportation-land use models with a spatial input-output structure. MEPLAN has been applied around the world for more than 20 years and is readily available for calibration; however, the Sacramento MEPLAN model is the first application in the United States. As an integrated land use and transportation model, MEPLAN is theoretically advanced. Changes in travel time and cost in the Sacramento MEPLAN model affect destination, mode, route, and location choices. The Sacramento MEPLAN model also represents land markets with endogenous prices as well as a redevelopment and demolition submodel. The mode choice model represents a relatively wide range of choices, including drive-alone, shared-ride, transit, and walk and bike modes. However, the Sacramento MEPLAN model's geographic representation is relatively coarse; it uses a sketch transportation network and 57 zones. The Sacramento MEPLAN model was calibrated by John Abraham and John D. Hunt as part of an urban model comparison project at the University of California at Davis and is suitable for academic research purposes.
Table 1
provides a summary of the core study policies that are examined alone and in different combinations in this study. These core study policies include transit and highway investment, auto pricing policies, and land use measures.
Table 1 Summary of Core Study Policies
Financially conservative expansion of the system; similar to a
3-year transportation improvement program.
591 new highway lane-miles, 6 new beltway interchanges,
service are added to LRT.
Restriction of development in slow and no-growth areas on the
The Base Case scenario described above is used in the sensitivity analysis. The Base Case scenario represents a financially conservative expansion of the Sacramento region's transportation system and is a point of comparison for the other scenarios examined in this study. This scenario would be close to a regional transportation improvement plan. It includes a relatively modest number of road-widening projects and new major roads, one freeway HOV lane segment, and a limited extension of light rail.
The alternative scenarios simulated for the sensitivity analyses represent plausible errors in projections of population, household income, and fuel price; they are presented in Table 2 .
Table 2 Summary of Scenarios for Sensitivity Analyses
|Household Income
|Fuel Price
|(percentage point change)
|(average annual growth)
|(average annual growth)
NOTE: One variable is varied at a time. The figures above for population and employment are percentage points, and for household income and fuel price are percentage change. The plausible error levels were identified in Rodier and Johnston (2002).
The percentage change in acres of land consumed in the sensitivity scenarios (compared to the Base Case scenario) is presented in Table 3 . Figures are provided for the entire region and by superzones, which are depicted in Figure 1 . The superzones represent important types of areas in the region and are useful categories for understanding the relative significance of changes in development patterns in the region. Researchers at the University of California at Davis and the University of Calgary developed these superzones in consultation with SACOG officials. The CBD superzone is the central business district in Sacramento. The inner suburbs of Sacramento, Citrus Heights and Roseville, and Rancho Cordova and Folsom superzones are important employment and housing centers in the region. The Citrus Heights and Roseville and the Rancho Cordova and Folsom superzones consist of relatively newer development than the inner suburbs. The outer ring generally consists of more agricultural and other environmentally sensitive lands, so less development is expected in this area.
Table 3 Percentage Change in Acres of Land Consumed by Superzone in the 2020
Sensitivity Scenarios Compared to the Base Case Scenario
|Sacramento CBD
|Citrus Heights/ Roseville
|Rancho Cordova/ Folsom
|Inner Suburbs
|Outer Ring
1 Error levels for projected annual population growth rates for California counties within 2 standard deviations; from
Rodier and Johnston 2002.
2 Error levels for household incomes for counties in the Sacramento region from CCSCE, 1997, as described in Rodier
and Johston 2002.
3 Error levels for fuel price.
We identified ±1.0 percent as a plausible confidence interval (1 standard deviation) for annual county population growth rate projections in the region. If the distribution of errors is normal, there is a 68 percent chance that the true value of the error will fall within this confidence interval and a 95 percent chance that it will fall within 2 standard deviations (±2.0 percent). As described above, we found evidence that the algebraic errors were distributed normally for the time intervals. Thus, for the Sacramento CBD, the 68 percent confidence interval for land
consumption is -0.5 to +0.4 percent; the 95 percent confidence interval is ±0.9 percent. For the Citrus Heights/Roseville superzone, the 68 percent confidence interval is -0.9 to
0.8 percent; the 95 percent confidence interval is -1.6 to 1.8 percent. For the Rancho Cordova/Folsom superzone, the 68 percent confidence interval is -1.2 to 1.6 percent; the 95 percent confidence interval is -2.3 to 3.2 percent. For the Inner Suburbs superzone, the 68 percent confidence interval is -0.5 to 0.6 percent; the 95 percent confidence interval is -1.0 to 1.3 percent. For the Outer Ring superzone, the 68 percent confidence interval is -0.5 to
0.7 percent; the 95 percent confidence interval is -1.0 to 1.4 percent. At the regional level, the 68 percent confidence interval is -0.6 to 0.7 percent; the 95 percent confidence interval is -1.1 to 1.5 percent.
The high-income scenario produced the largest increase in land consumption. This increase ranged from a low of 1.5 percent in the Sacramento CBD to a high of 3.9 percent in the Rancho Cordova/Folsom superzone. At the regional level, the high-income scenario increases land consumption by 3.1 percent. The fuel price sensitivity scenarios produced little change in land consumption. The largest decrease was 0.1 percent in the high and highest fuel price scenarios in the Sacramento CBD. The largest increase was 0.1 percent in the Outer Ring in the highest fuel price scenario. At the regional level, the change was negligible.
The level of variation produced from the population and income sensitivity scenarios by superzone and at the regional level is relatively large; however, it is relatively small for the fuel price sensitivity scenarios.
The travel and emission results of the sensitivity analyses are presented in
Table 4
. In the population sensitivity scenarios, the 68 percent confidence interval for VMT is -2.5 to +2.9 percent; the 95 percent confidence interval is -5.3 to +6.7 percent. For vehicle trips, the 68 percent confidence interval is -2.7 to +3.5 percent; the 95 percent confidence interval is -5.6 to +6.8 percent. For vehicle travel speed, the 68 percent confidence interval is 0.0 to +0.9 percent; the 95 percent confidence interval is -0.8 to +1.8 percent. The greatest variation in vehicle emissions projections was obtained for total organic gases (TOG). The 68 percent confidence interval for TOG is -2.4 to +3.5 percent; the 95 percent confidence interval is -6.7 to +8.2 percent. The smallest variation in vehicle emission projections was obtained for NOx. The 68 percent confidence interval for NOx is -2.3 to +3.0 percent; the 95 percent confidence interval is -5.3 to +6.8 percent. The level of variation produced from the population, income, and fuel price sensitivity scenarios for vehicle travel and emissions results are relatively moderate.
Table 4 Percentage Change in Daily Travel and Emissions Results in the 2020 Sensitivity Scenarios Compared to the Base Case Scenarios
VEHICLE TRAVEL VEHICLE EMISSIONS (tons) Mean TRIPS VMT TOG CO NOx PM
Speed BASE CASE 5.4 million 44.7 million 33 mph 14.2 124.4 55.1 84.6 POPULATION1 Lowest -5.6% -5.3% 1.8% -6.7% -6.0% -5.3% -7.3% Low -2.7% -2.5% 0.9% -2.4% -2.4% -2.3% -2.1% High 3.5% 2.9% 0.0% 3.5% 3.1% 3.0% 3.3% Highest 6.8% 6.7% -0.8% 8.2% 7.1% 6.8% 7.1% HOUSEHOLD INCOME2 High 9.7% 3.1% -1.3% 2.3% 3.9% 3.1% 3.1% FUEL PRICE3 Low -0.4% -3.3% 2.1% -8.2% -7.7% -6.9% -8.4% Moderate -0.1% -3.0% 2.3% -8.8% -8.0% -7.0% -9.2%
1 Error levels for projected annual population growth rates for California counties within 2 standard deviations.
2 Error levels for household incomes for counties in the Sacramento region from CCSCE, 1997.
3 Error levels for fuel price from the EIA, 2001.
Land use results for the other scenarios are discussed in comparison to the future Base Case scenario, except where noted. Table 5 presents the household and employment results by superzone for the Base Case scenario, and the change for alternative scenarios. Table 6 presents the total developed acres of land by superzone for the Base Case Scenario, and the change for alternative scenarios.
Table 5 Percentage Change in 2020 Household and Employment Activities
Compared to the Base Case Scenario
Outer Ring
|Advanced LRT
|Highest VMT Pricing
|Advanced LRT+ Low VMT Pricing
|Urban Reserve + Infill + Advanced LRT
|Urban Reserve + Infill + Advanced LRT + Lowest VMT Pricing
|UGB + Advanced LRT
|UGB + Advanced LRT + Lowest VMT Pricing
|Advanced LRT
|Highest VMT Pricing
|Advanced LRT +Low VMT Pricing
|Urban Reserve + Infill + Advanced LRT
|Urban Reserve + Infill + Advanced LRT + Lowest VMT Pricing
|UGB + Advanced LRT
|UGB + Advanced LRT + Lowest VMT Pricing
|Table 6
|Percentage Change in 2020 Land Consumption (Acres)
|Compared to the Base Case Scenario
|Sacramento CBD
|Citrus Heights/ Roseville
|Rancho Cordova/ Folsom
|Inner Suburbs
|Outer Ring
Table 6 Percentage Change in 2020 Land Consumption (Acres)
Compared to the Base Case Scenario (Continued)
Outer Ring TOTAL
Highest VMT Pricing 0.5% 1.0% 1.7% 0.7% 0.2% 0.5% Advanced LRT+Low VMT
Pricing 0.5% 0.4% 1.1% 0.3% 0.0% 0.2%
Urban Reserve + Infill + 0.1% -0.5% -0.2% -0.5% -6.2% -4.1%
Advance LRT
Urban Reserve + Infill +
Advanced LRT + Lowest 0.2% -0.3% 0.2% -0.4% -6.1% -4.0%
VMT Pricing
UGB + Advanced LRT 1.1% 1.5% 2.8% 0.8% -9.8% -5.8%
UGB + Advanced LRT +
Lowest VMT Pricing 1.2% 1.8% 3.3% 0.9% -9.8% -5.6%
In both the HOV and Beltway scenarios, there is an increase in the shared-ride mode share compared to the Base Case scenario. The mode share results are presented in Table 7 . Faster travel speeds resulting from the HOV lanes in the HOV and Beltway scenarios make carpooling more attractive than most of the other available modes, and there is a reduction in the drive-alone, walk, and bike mode shares. Transit mode share is reduced in the Beltway scenario and slightly increased in the HOV scenario. The last result may be due to faster travel times by buses that take advantage of HOV lanes.
Table 7 Percentage Change in 2020 Daily Mode Share Results
Compared to the Base Case Scenario
BASE CASE 45.1 43.8 1.8 9.3 HOV -4.3%1 5.5% 0.6% -5.5% Beltway -4.3% 6.2% -1.7% -7.8% LRT -1.6% -1.1% 86.7% -3.9% Advanced LRT -7.4% -7.0% 433.2% -15.9% Highest VMT Pricing -9.5% 8.1% 23.8% 3.3% Advanced LRT+ Low VMT
-11.8% -4.5% 474.0% -13.7%
Urban Reserve + Infill +
-6.6% -7.2% 385.6% -9.2%
Advanced LRT
Urban Reserve + Infill +
Advanced LRT + Lowest VMT -9.3% -6.1% 419.3% -8.1%
1 Figures are percentage change from the Base Case.
Table 7 Percentage Change in 2020 Daily Mode Share Results Compared to the Base Case Scenario (Continued)
UGB + Advanced LRT +
Lowest VMT Pricing -9.9% -7.5% 487.9% -11.2% 1 Figures are percentage change from the Base Case.
In the LRT and Advanced LRT scenarios, the light rail and advanced transit service investments and a modest increase in the intensity of activities along light rail lines result in faster transit travel times and produce relatively large gains in the transit mode share and losses in the drive-alone, shared-ride, and walk and bike mode shares.
The highest VMT pricing scenario produces a strong reduction in the drive-alone mode share. There are large increases in the modes for which the pricing charges do not apply (transit, walk, and bike modes) or are lower (shared-ride). This is the only scenario that increases walk and bike share because in this scenario the walk and bike modes are free and there is no improvement to alternatives modes without charges (for example, transit) or with lower charges (for example, HOV). When the Advanced LRT network is added to the low VMT pricing policies, there is a larger increase in the transit mode share and a reduction in the drive-alone, shared-ride, and walk and bike mode shares. The new transit service improves the relative accessibility of the transit mode compared to the other modes in this scenario.
In the Urban Reserve, Infill Subsidy, and Advanced LRT scenario, transit mode share is increased significantly compared to the Base Case scenario and the LRT scenario. Again, the drive-alone, shared-ride, and walk and bike mode shares are all reduced in this scenario compared to the Base Case. When the lowest VMT pricing policy is added to this scenario, we again see large reductions in the drive-alone mode share and large increases in the shared-ride, transit, and walk and bike mode shares. However, the increase in the transit share in this scenario is somewhat smaller than that obtained from the Advanced LRT-only scenario. It appears that the distribution of protected lands and the infill subsidy policy in this scenario do not successfully promote land uses that support transit use.
In the UGB and Advanced LRT scenario, there are large increases in the transit mode share and reductions in the drive-alone, shared-ride, and walk and bike mode shares compared to the Base Case scenario. It is difficult to represent the effect that land use measures (UGB and infill subsidy), which would most likely be combined with urban design policies such as improved bike and pedestrian connectivity, could have on the walk and bike mode share because the Sacramento MEPLAN model uses large zones and does not explicitly include variables that represent the walkability and bikeability of neighborhoods. As a result, the walk and bike mode share may be underestimated in the analysis of land use measures.
When the lowest VMT pricing policy is added to the UGB and Advanced LRT scenario, the reduction in the drive-alone mode share is significantly increased, and transit mode share is larger compared to both the base case and the UGB and Advanced LRT scenarios. The increase in the per mile cost of auto travel is a disincentive to driving alone, and a relative reduction in the cost of travel by transit increases the transit mode share. The shared-ride and walk and bike mode shares are also reduced in this scenario.
In the HOV and Beltway scenarios, the HOV lanes provide faster travel times for shared-ride vehicles to produce larger shared-ride and smaller drive-alone mode shares; thus there is a modest decrease in vehicle trips. Vehicle travel results are presented in Table 8 . Despite these mode shifts, increased auto travel speeds and decentralization of employment and household activities produce longer trips and increased VMT.
Table 8 Percentage Change in 2020 Daily Vehicle Travel Results
Compared to the Base Case Scenario
Trips VMT Travel Speed
BASE CASE 5.41 million 44.8 million 33 mph
Beltway -1.0% 9.6% 2.5%
Advanced LRT -7.3% -6.0% 3.5%
Highest VMT Pricing -4.2% -10.0% 2.0%
Low VMT Pricing + Advanced LRT -9.1% -13.0% 3.7%
Urban Reserve + Infill + Advanced LRT -7.5% -8.8% 3.3%
Urban Reserve + Infill + Advanced LRT +
-8.7% -12.9% 3.5%
Lowest VMT Pricing
UGB + Advanced LRT -8.7% -10.2% 4.0%
UGB + Advanced LRT + Lowest VMT Pricing -9.5% -13.7% 4.0%
1 Figures are percentage change from the base scenario
In the LRT and Advanced LRT scenarios, increased transit accessibility and a modest centralization of activities shift trips from the auto to transit and reduce VMT. The costs imposed on the auto modes in the highest VMT pricing scenario produce reductions in auto trips and VMT and increase auto travel speeds. When the Advanced LRT network is added to the low VMT pricing policy, the reduction in vehicle trips and VMT is increased, and the increase in auto travel speeds is greater.
The Advanced LRT scenario and the low VMT pricing and Advanced LRT scenario produce increases in auto travel speeds that are greater than both the HOV and the Beltway scenarios. The highest VMT pricing-only scenario produces an increase in travel speed that is greater than the HOV lane scenario and almost as great as the Beltway scenario.
Adding the urban reserve and infill subsidy measures to the Advanced LRT policy produces somewhat larger reductions in vehicle trips and VMT, but not greater increases in auto travel speed. The addition of the VMT pricing policy significantly improves these results, but not compared to the Low VMT pricing and Advanced LRT scenario because of the difference in the VMT pricing level (that is, it is lower).
In the UGB and Advanced LRT scenario, there is an increase in the reduction of vehicle trips, VMT, and auto travel speed compared to the Advanced LRT-only scenario. The increase in vehicle travel speed is higher than that obtained for the HOV and Beltway scenarios. When the lowest VMT pricing policy is added to the scenario, the reduction in vehicle trips and VMT is increased. However, these reductions are only somewhat larger than the results for the low VMT pricing and Advanced LRT scenario. Again, this is because of the difference in the pricing levels for these policies. The increase in auto travel speeds in this scenario is also greater than the results for the HOV and Beltway scenarios.
The daily emissions results are presented in Table 9 . The emissions results generally follow from the travel results. The HOV and Beltway scenarios increase vehicle emissions. The increase in emissions for the Beltway scenario is relatively large. All the other scenarios result in a reduction in emissions. The Lowest VMT pricing, UGB, and Advanced LRT scenario produces the greatest reduction in emissions, followed by the Low VMT pricing and Advanced LRT scenario; the Lowest VMT pricing, Urban Reserve, Infill Subsidy, and Advance LRT scenario; the UGB and Advanced LRT scenario; the Highest VMT pricing scenario; the Urban Reserve, Infill Subsidy, and Advanced LRT scenario; the Advanced LRT scenario; and finally the LRT only scenario.
Table 9 Percentage Change in 2020 Daily Emissions (Tons) Results
Compared to the Base Case Scenario
BASE CASE 14.2 124.4 55.1 84.6 HOV 2.1%1 1.4% 0.9% 0.3% Beltway 6.3% 8.7% 8.5% 2.9% LRT -4.0% -2.9% -2.0% -4.9% Advanced LRT -11.2% -8.4% -5.7% -12.8% Highest VMT Pricing -13.4% -10.9% -8.9% -14.3% Low VMT Pricing + Advanced
-18.4% -15.0% -12.0% -19.9%
Urban Reserve + Infill +
-12.6% -9.4% -6.8% -13.3%
Advanced LRT
Urban Reserve + Infill +
Advanced LRT + Lowest VMT -17.9% -14.9% -12.1% -19.5%
UGB + Advanced LRT -15.1% -12.3% -9.4% -17.3%
UGB + Advanced LRT + Lowest
VMT Pricing -19.1% -15.8% -12.9% -21.0%
1 Figures are percentage change from the base scenario.
These levels of variation are relatively large; land consumption results from transportation investment and auto pricing scenarios in the alternative scenario analysis do not typically fall outside of the ±1.0 percent range. The exceptions are the scenarios that include the land use measures. For VMT, the range is -2.5 to 2.9 percent at the 68 percent confidence level and -5.3 to 6.7 percent at the 9 percent confidence level. For TOG emissions, the range is -2.4 to
3.5 percent at the 68 percent confidence level and -6.7 to 8.2 percent at the 95 percent confidence level.
The level of variation produced from the population, income, and fuel price sensitivity scenarios for vehicle travel and emissions results are relatively moderate. The vehicle travel and emission results for the alternative scenario analysis typically fall outside of the confidence intervals for errors in population, income, and fuel price projections. The exceptions are some of the more moderate auto pricing scenarios and the transportation investment scenarios (the HOV and the LRT-only scenarios). If the results of an alternative scenario do not fall outside the confidence intervals established in the sensitivity analysis of errors in population, income, and fuel price projections for the Base Case scenario, the results for the alternative scenario cannot be considered significantly different from the Base Case scenario.
I. Introduction
Hello, my name is ____. I am involved a California Department of Transportation and Mineta Transportation Institute study that examines potential improvements to urban and regional models and their application in the policy process. As part of this study, we are conducting interviews with experts on their experiences and opinions with urban and regional models. Your name has been provided to me as a leading expert in the field. Would you be willing to be interviewed? Interview responses will be reported in the aggregate and your name would be kept confidential.
II. Preliminary Information
III. Experience/Background of Interviewee
IV. Modeling Improvements
availability of data, regulatory requirements, political environment, funding, and availability of trained staff)?
V. Modeling Problems
VI. Modeling State of the Practice and Art
VII. Communication of Uncertainty
1. As you know, there has been a lot of research on modeling uncertainty in the last five years. What are your thoughts on the prospects of communicating more complex results including land use, cost benefit analyses, and confidence intervals on key outputs to the public and policy makers? Feasibility? Best practices? Success stories?
VIII. Other experts
1. Can you recommend anyone else we may want to interview for this study?
Thank you very much for participating in this study. I really appreciate your time.
54. Rodier and Johnston, 2002.
55. Rodier, 2004.
56. Rodier, 2006.
57. Norman Marshall and Brian Grady, "Travel Demand Modeling for Regional Visioning and Scenario Analysis," Transportation Research Record 1921 (2005): 44-52.
58. Ibid., 45.
59. Ibid., 48.
60. Ibid., 51-52.
61. Appendix A .
62. Fotis G. Stathopoulos and Robert B. Noland, "Induced Travel and Emissions from Traffic Flow Improvement Projects," Transportation Research Record 1842 (2003): 57.
63. Steven W. Popper, Robert J. Lempert, and Steven C. Banks, "Shaping the Future," Scientific American: 67-71. April 2005.
64. Ibid., 69.
65. Ibid., 70.
66. Ibid., 70.
|Algebraic error
|Algebraic percent error
|Central Business District
|Carbon monoxide
|Carbon dioxide
|Department of Transportation
|Environmental Protection Agency
|High-occupancy vehicle
|Intermodal Surface Transportation Efficiency Act
|Light Rail Transit
|Global information system
|Mean algebraic percent error
|Mean absolute percent error
|Metropolitan planning organization
|Mineta Transportation Institute
|National Environmental Policy Act
|Nitrous oxides
|Sacramento Area Council of Governments
|Single-occupant vehicle
|Standard deviation
|Transit-Oriented Development
|Total organic gases
|Urban growth boundary
|Vehicle hours of delay
|Vehicle hours traveled
|Vehicle miles traveled
Cambridge Systematics. "Wasatch Frong Regional Council (WFRC) Model Sensitivity Testing and Training Study." Utah Department of Transportation, 2003.
Chu, Xuehao. "Highway Capacity and Areawide Congestion." Preprint for the 79th Annual Meeting of the Transportation Research Board at the National Research Council. Washington DC, 2000.
Clay, Michael J., and Robert A. Johnston. "Multivariate Uncertainty Analysis of an Integrated Land Use and Transportation Model: MEPLAN." Transportation Research D 11(3) (2005): 191-203.
DeCorla-Souza, Patrick, and Harry Cohen. "Accounting for Induced Travel in Evaluation of Metropolitan Highway Expansion." Preprint for the 77th Annual Meeting of the Transportation Research Board at the National Research Council. Washington DC, 1998.
Fulton, Lewis M., Daniel J. Meszler, Robert B. Noland, and John V. Thomas. "A Statistical Analysis of Induced Effects in the U.S. Mid-Atlantic Region." Preprint for the 79th Annual Meeting of the Transportation Research Board at the National Research Council. Washington DC, 2000.
Fyvbjerg, Bent, Mette Skamris, and Soren L. Buhl. "How (In)accurate Are Demand Forecasts in Public Works Projects?" Journal of the American Planning Association 71(2), Spring 2005.
Goodwin, Phil B. "Empirical evidence of induced traffic, a review and synthesis." Transportation 23 (1996): 35-54.
Goodwin, Phil, and Robert Noland. "Build new roads really does create extra traffic: a response to Prakash, et al." Applied Economics 35 (2003): 1451-1457.
Hansen, Mark, and Yuanlin Huang. "Road supply and traffic in California urban areas." Transportation Research A 31 (1997): 205-218.
Harvey, Greig, and Elizabeth Deakin. "Description of the STEP Analysis Package." Berkeley, CA, 1995.
Krishnamurthy, Sriram, and Kara M. Kockelman. "Propagation of Uncertainty in Transportation Land Use Models: Investigation of DRAM-EMPAL and UTPP Predictions in Austin, Texas." Transportation Research Record 1831 (2003): 219.
Marshall, Norman, and Brian Grady. "Induced Travel Results." Resource Systems Group for the U.S. Environmental Protection Agency, 2001.
_____. "Travel Demand Modeling for Regional Visioning and Scenario Analysis." Transportation Research Record 1921 (2005): 44-52.
Noland, Robert B., and William A. Cowart. "Analysis of Metropolitan Highway Capacity and the Growth in Vehicle Miles of Travel." Preprint for the 79th Annual Meeting of the Transportation Research Board at the National Research Council. Washington DC, 2000.
Pickrell, Don H. Urban Rail Transit Projects: Forecast Versus Actual Ridership and Costs. Final Report. U.S. Department of Transportation. Washington DC, 1990.
Popper, Steven W., Robert J. Lempert, and Steven C. Banks. "Shaping the Future." Scientific American: 67-71. April, 2005.
Pradhan, Anant, and Kara Maria Kockelman. "Uncertainty Propogation in an Integrated Land Use–Transportation Modeling Framework: Output Variation via UrbanSim." Transportation Research Record 1805 (2002): 128-135.
Richmond, Jonathan E. D. "New Rail Transit Investments–A Review." John F. Kennedy School of Government, Harvard University. Cambridge, MA, June 1998.
Rodier, Caroline J. "A Case Study of Induced Travel in the Sacramento Region Using a Travel Demand Model and an Integrated Land Use and Transportation Model. A Report for the Environmental Protection Agency." December, 2002.
_____. Verifying the Accuracy of Regional Models Used in Transportation and Air Quality Planning. San Josè, CA: Mineta Transportation Research Institute. MTI Report 02-03, June 2003.
_____. "Verifying the Accuracy of Regional Models Used in Transportation and Air Quality Planning." Transportation Research Record 1898 (2004): 45-51.
_____. Verifying the Accuracy of Integrated Land Use and Transportation Models Used in Transportation and Air Quality Planning: A Year Two Study. San Josè, CA: Mineta Transportation Research Institute, 2006.
Rodier, Caroline J., John E. Abraham, Robert A. Johnston, and John D. Hunt. "Anatomy of Induced Travel Using an Integrated Land Use and Transportation Model in the Sacramento Region." Preprint for the Annual Meeting of the Transportation Research Board. Washington DC, 2001.
Rodier, Caroline J., and Robert A. Johnston. "Uncertain socioeconomic projections used in travel demand model and emissions models: could plausible errors result in air quality nonconformity?" Transportation Research A 36 (2002): 613-631.
Standing Advisory Committee on Truck Road Assessment (SACTRA). "Truck Roads and the Generation of Traffic." London: Department of Transport, HMSO, 1994.
Stathopoulos, Fotis G., and Robert B. Noland. "Induced Travel and Emissions from Traffic Flow Improvement Projects." Transportation Research Record 1842 (2003): 57-63.
Waddell, Paul. "UrbanSim-Historic Validation in Eugene/Springfield," Second Oregon Symposium on Integrating Land Use and Transport Models, July (2000).
Zhao, Yong, and Kara M. Kockelman. "The propagation of the uncertainty through travel demand forecasts: An exploratory analysis." In Annals of Regional Science (2002) 36:145-163.
Caroline Rodier is an assistant research engineer at the University of California, Berkeley. Her research interests include transportation policy and planning; land use and travel demand modeling; the behavior effects of new technology applications in transportation; and elderly travel behavior. Some of her current projects include analyses of modeling uncertainty in the context of environmental impact and air quality conformity processes; a behavioral evaluation of a transit-based smart parking field test; a feasibility analysis of a virtual compliance station for commercial vehicles; and social marketing interventions to enhance older traveler mobility. She has authored 16 journal articles, 29 proceedings articles, and 22 research reports.
San José State University, of the California State University system, and the MTI Board of Trustees have agreed upon a peer view process to ensure that the results presented are based upon a professionally acceptable research protocol.
Research projects begin with the approval of a scope of work by the sponsoring entities, with in-process reviews by the MTI research director and the project sponsor. Periodic progress reports are provided to the MTI research director and the Research Associates Policy Oversight Committee (RAPOC). Review of the draft research product is conducted by the Research Committee of the board of trustees and may include invited critiques from other professionals in the subject field. The review is based on the professional propriety of the research methodology.