THE GALLUP ORGANIZATION

for
U.S. Department of Transportation
Federal Highway Administration (FHWA)

FHWA Study — Results
Abridged Final Report

Implementing Performance Measurement in Environmental Streamlining

Submitted to:

Federal Highway Administration
Planning and Environment, 3222 HEPE
400 7th Street, SW
Washington, D.C. 20590

January 2004

Prepared by:

Alison Simon, Ph.D., AICP
Sameer Abraham, Ph.D.
Calvin Jones, Ph.D.
The Gallup Organization
Government Division
901 F Street, NW
Washington, D.C. 20004

FOREWORD

This report is the culmination of a two year effort to establish a baseline assessment of the Federal Highway Administration environmental streamlining program. FHWA is pleased to sponsor this work as an outcome of the streamlining efforts mandated by the Transportation Equity Act for the 21st Century (TEA-21) of 1998.

FHWA intends to publish excerpts and reports from this document over the next few months on their Internet site. We encourage individuals to keep track of these publications on our website at http://www.fhwa.dot.gov.

ACKNOWLEDGEMENTS

The authors would like to thank those who contributed to the production of this report: Kreig Larson who served as the government project officer, and Fred Skaer, Director of Project Development and Environmental Review Office, who provided guidance and support throughout the process.

The National Technical Review Board also provided feedback and encouragement throughout the process. This was crucial to ensuring the long-term feasibility of the project. The board members:

Ms. Janine Bauer, Coalition to Defend NEPA
Mr. John Devierno, Montana Department of Transportation
Ms. Robin Elliott, U.S. Fish & Wildlife Service
Mr. Marlin Gottschalk, Senior Policy Advisor, Georgia Department of Natural Resources
Ms. Dorothy Guzzo, Deputy Historic Preservation Officer, New Jersey State Historic Preservation Office
Ms. Julie Hunkins, North Carolina Department of Transportation
Mr. Hal Kassof, Vice President, Highway Program Area Manager, Parsons Brinckerhoff
Ms. Susan McDonald, Pennsylvania Department of Transportation
Mr. Gary McVoy, Director of Transportation Maintenance, New York State Department of Transportation
Ms. Anne Miller, Director, Office of Federal Activities, Environmental Protection Agency
Mr. Sam Morigeau, U.S. Department of Agriculture
Mr. Joe Ossi, Federal Transit Administration
Ms. Nancy Schamu, National Conference of State Historic Preservation Officers
Mr. Vaughn Stokes, Director of Engineering, U.S. Department of Agriculture
Ms. Gina Whitehill-Baziuk, Metro
Ms. Sandy Straehl, Chief, Programs of Policy & Analysis, Montana Department of Transportation
Mr. Benjamin Tuggle, Chief, Division of Habitat Conservation, U.S. Fish & Wildlife Service
Ms. Janet D'Ignazio, Chief Planning & Environmental Officer, North Carolina Department of Transportation
Mr. Michael Ryan, Deputy Secretary for Highway Administration, Pennsylvania Department of Transportation
Ms. Susan Borinsky, Director, Office of Human & Environment, Federal Transit Administration
Mr. Richard Brandman, Metro
Mr. Brent Felker, Chief Engineer, California Department of Transportation
Mr. Gary Gallegos, Executive Director, San Diego Association of Governments
Mr. Doug Allen, Dallas Area Rapid Transit
Ms. Dinah Bear, General Counsel, Council on Environmental Quality
Mr. John Studt, Chief, Regulatory Branch, Corps of Engineers
Ms. Bambi Kraus, President, National Association of Tribal HPO Officers
Mr. Nicholas Mpras, Chief, Office of Bridge Administration, U.S. Coast Guard
Mr. Jonathan Deason, American Road and Transportation Builders Association
Ms. Sara Campbell, Surface Transportation Policy Project
Mr. Dave Burwell, Surface Transportation Policy Project
Ms. Nancy Jakowitsch, Director, Smart Growth & International Programs, Surface Transportation Policy Project
Ms. Patricia White, Transportation Associate, Defenders of Wildlife

Contents

• Executive Summary
• Overview
• Research Objectives and Study Design
• Specifying the Population of Interest
• Data Collection
• Follow-Up Procedures
• The 2003 FHWA Survey of Transportation Agencies and Resource-Permitting Organizations
• Report of Major Findings
• Recommendations for Management

Exhibits & Charts

Exhibit 1. Distribution of Reviewers and Managers by Type of Organization

Exhibit 2. Recency of Experience of Reviewers with NEPA Projects by Type of Organization

Exhibit 3. Types of Projects Evaluated by Reviewers by Type of Organization

Exhibit 4. Comparison of Mean (Average) Satisfaction Scale Scores Given by Transportation and Resource Reviewers to Their Named Counterpart Organizations

Exhibit 5. Stage of NEPA Project at which Problems Reported by Reviewers Occurred

Exhibit 6. Problems Experienced by Reviewers on NEPA Projects by Type of Organization

Exhibit 7. Reviewers' Perceptions of the Relationships, Communications, and Timeliness of Interactions Between Transportation and Resource-Permitting Organizations

Exhibit 8. Ratings by Reviewers of the Performance of Named Counterpart Organizations on Cited NEPA Projects

Exhibit 9. Perceptions of Managers and Reviewers of the General Performance of Named Counterpart Organizations on all NEPA Projects on Which They Have Interacted

Exhibit 10. Comparison of Mean (Average) Agree/Disagree Scale Scores for Counterpart Organization Ratings

Exhibit 11. Overall Evaluations by Transportation and Resource Officials of the Current Relationship with Their Named Counterpart Organizations

Exhibit 12. Change in Relationships with Their Named Counterpart Organizations Perceived by Transportation and Resource Officials during the Last Three Years

Chart 1. Overall Satisfaction Expressed by Transportation Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Chart 2. Overall Satisfaction Expressed by Resource Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Chart 3. Transportation Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Chart 4. Resource Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Executive Summary

During 2003, the Federal Highway Administration (FHWA) contracted with The Gallup Organization to conduct a nationwide survey of the perceptions of stakeholders and participants in the environmental streamlining process. Telephone interviews were conducted with over 700 senior officials in Federal, State, and local government transportation agencies and with more than 600 senior officials in both government and private resource-permitting organizations. All survey participants were either agency managers or senior technical staff directly involved in the completion of environmental review of large scale transportation projects under the requirements of the National Environmental Policy Act of 1969, as amended (NEPA).

Survey questions focused on the views of managers and technical staff in both types of organizations concerning their overall relationships with counterpart organizations. Technical staff members in both types of agencies were also asked about their assessments of the performance of counterpart organizations on recently completed transportation projects.

The primary intent of the survey was to differentiate between the aspects of the relationships and interactions between transportation agencies and resource organizations that are working smoothly and productively and those that are viewed as needing improvement to enhance the efficiency of the NEPA process. A second key goal was to establish a baseline measure of stakeholder perceptions against which data from future surveys might be compared to assess progress in inter-agency relationships and performance.

The Technical Reviewers' Perspective

Technical staff members who served as Reviewers on recently completed NEPA projects were asked a series of 40 questions about specific dimensions of their experiences on a recently completed NEPA project on which they had the most interaction with counterpart organizations. (Transportation agency Reviewers were asked for their perceptions about resource-permitting organizations and vice versa.) Technical Reviewers were also asked for an overall assessment of their satisfaction with the performance of their designated counterpart organization on all aspects of that project.

Reviewers in both transportation and resources organizations answered most of the survey questions about project interactions in very similar ways. The pattern of responses to the question about overall satisfaction (shown below) typifies the results for most of the questions about project interactions.

Approximately two-thirds of the Reviewers in both transportation agencies (67 percent) and resource organizations (64 percent) reported that they were either satisfied or very satisfied with the performance of their counterparts on the NEPA project. One fifth (20 percent) of the Reviewers in both types of organizations gave a neutral response (neither satisfied nor dissatisfied). Only one eighth (12 percent) of Reviewers in transportation agencies and one-sixth (16 percent) of those in resource organization evinced dissatisfaction or strong dissatisfaction with their counterparts' performance.

The responses to the overall satisfaction question correlated strongly with answers to another question asking Reviewers to report on any problems experienced on the recently completed NEPA project they cited — just over one-third of both types of Reviewers reported having problems with their named counterpart organization. Reviewers who reported problems were far more likely than those who did not report problems to indicate that they were dissatisfied with their counterparts' performance or to give a neutral response to the overall satisfaction question.

Scope of Problems Raised

Of the transportation Reviewers who reported problems with their resource organization counterparts, nearly two-thirds indicated that the problems occurred during the analysis of impacts and during the finalizing documents or responses to comments on their NEPA projects. About half indicated that problems arose during the process of obtaining commitment to mitigation measures.

Of the resource Reviewers who reported problems with their transportation counterparts, about two-thirds said that the problems occurred during the analysis of impacts (mirroring the views of transportation Reviewers), and also during the collection of data and information. Over half of resource Reviewers also reported problems during the finalizing of documents or responses to comments (also matching reports by transportation Reviewers) and the development and analysis of alternatives.

The most common type of problem cited by transportation Reviewers was "lack of timely response" by the counterpart resource agency, mentioned by 21 percent. Resource Reviewers gave even more varied descriptions of project problems, with 11 to 15 percent citing "poor communication," "poor coordination," "receiving wrong or incomplete information," as well as "lack of timely response."

Response Patterns of Project Reviewers to Survey Questions About Agency Interactions and Relationships

Response patterns expressed by both types of Reviewers on several dimensions of agency interactions on the recently completed NEPA project closely resembled the distribution of answers to the "overall satisfaction" question, including:

• 5 of 8 questions about the tenor of the relationship between the counterpart organizations on the NEPA project, specifically:
  • whether the counterpart organization provided materials, information, or documentation that the respondent's agency needed;
  • whether the contribution of the respondent's organization was appreciated by the counterpart agency;
  • whether the counterpart organization wanted to play an active role in the NEPA project;
  • whether the respondent's agency's opinions seem to count in the process; and
  • whether the counterpart organization felt the mission of the project was important;

• 6 of 11 survey items on the quality and character of communications with counterpart organizations on the NEPA project, namely:
  • whether the counterpart organization involved the respondent's agency early in the project;
  • whether the counterpart organization responded in a timely way to the respondent agency's requests;
  • whether the counterpart organization was open and honest with the respondent's agency;
  • whether the counterpart organization was open to suggestions and alternatives;
  • whether the counterpart organization gave clear explanations if they did not agree with the respondent agency's proposals; and
  • whether there was a sufficient level of communication between the two agencies on the project.

• 3 of 4 questions about the timeliness of performance by the counterpart organization, including:
  • whether the counterpart organization adhered to schedules throughout the project;
  • whether the counterpart organization gave the respondent's agency enough time to accomplish tasks; and
  • whether the entire process took a reasonable amount of time.

Differences in Opinions Between Reviewers

A number of noteworthy exceptions to the response pattern described above were found among the evaluations given by technical review staff in each of the three aspects of project interactions. These exceptions include some noteworthy differences in opinions on key issues:

• On inter-agency relationships:
  • Nearly all Reviewers of both groups indicated that their own agencies understood what was expected of them on the NEPA project;
  • Resource organization Reviewers were substantially more likely than transportation agency Reviewers to agree that the counterpart organization helped to move the project forward (75 percent for resource Reviewers vs. only 56 percent for transportation Reviewers);
  • Transportation Reviewers were much less likely than resource Reviewers to agree that their counterpart organization made efforts to improve the process during the NEPA project (36 percent for transportation Reviewers vs. 57 percent for resource Reviewers).

• With respect to inter-agency communications:
  • resource Reviewers were considerably more likely than transportation Reviewers to agree that the counterpart organization invited their participation in key meetings (71 percent for resource Reviewers vs. 54 percent for transportation Reviewers);
  • transportation Reviewers were far less likely than resource Reviewers to agree that the counterpart organization kept them informed about their progress (41 percent of transportation Reviewers vs. 58 percent of resource Reviewers);
  • both groups of Reviewers were less likely to agree that their counterpart organizations
    • gave reasonable suggestions or alternatives on the NEPA project; or
    • were willing to compromise on the project;
  • resource Reviewers were substantially more likely than transportation Reviewers to agree that their counterpart organization had adequate participation in key meetings (84 percent for resource Reviewers vs. 65 percent for transportation Reviewers);

• For the questions about timeliness:
  • resource Reviewers were far less likely than transportation Reviewers to agree that the project could have been shortened without compromising the intent of NEPA (35 percent for resource Reviewers vs. 55 percent for transportation Reviewers).

Responses to Survey Questions about Agency Performance

Transportation and resource Reviewers were asked 8 additional questions about the characteristics and performance of their counterpart agencies on the recently completed NEPA project. On 5 of the 8 questions, the response patterns from both groups of reviewers remained similar to and highly correlated with the answers they gave to the overall satisfaction question (noted in Table I), including satisfaction with:

• the quality of information provided by the counterpart organization;
• the completeness of information provided by the counterpart organization;
• the counterpart organization's ability to stay organized throughout the project;
• the counterpart organization's understanding of the respondent agency's mission; and
• the counterpart organization's willingness to consider a range of mitigation measures.

Significantly different response patterns were found on 3 of the survey questions:

• Reviewers from both groups of organizations were much more likely than average to indicate satisfaction with the competence of the counterpart agency staff they interacted with;
• Reviewers from both groups of organizations were much less likely than average to indicate satisfaction with the range of reasonable alternatives their counterparts suggested for the project;
• Resource Reviewers were much more likely than transportation Reviewers to express satisfaction with the level of resources the counterpart organization devoted to the project (70 percent of resource Reviewers vs. only 48 percent of transportation Reviewers).

When asked to report their satisfaction with the job done by their transportation counterparts in protecting the environment, the response pattern was nearly identical to that given to the overall satisfaction question described above (see Table I).

The Perspective of Agency Managers and Reviewers on General Agency Relationships

Senior Managers and technical Reviewers in both transportation agencies and resource organizations were asked to make an additional series of 8 general evaluations of their named counterpart organizations — these evaluations were to be broader than those provided to the preceding questions about the recently completed NEPA project. These high-level evaluation questions asked whether:

1. the counterpart organization understands your agency's mission;
2. the counterpart organization cares about your agency's mission;
3. the counterpart organization is committed to doing quality work;
4. the counterpart organization has competent staff;
5. there is a sufficient level of trust between your two agencies;
6. the counterpart organization is committed to making the environmental review process run efficiently;
7. the counterpart organization is willing to compromise; and
8. there is a sufficient level of communication between your two agencies.

In addition, transportation Managers and Reviewers were asked whether their resource counterpart organization is committed to transportation improvements, while resource Managers and Reviewers were asked whether their transportation counterpart organization is committed to protecting the environment.

For items numbered 1, 3, 4, and 8 in the preceding list, most groups of respondents exhibited answer patterns that closely matched the distributions observed for the "overall satisfaction" question (shown in Table I). However, on the first of these questions (whether the counterpart understands the respondent's agency's mission), transportation Managers were much less likely than the other three types to express agreement (only 48 percent agreed or strongly agreed).

Response patterns to the remaining four items (2, 5, 6, and 7 in the list above) were considerably different from the overall satisfaction pattern (see Table I). In general, all four groups of respondents were less likely to agree with these statements about inter-agency relationships. Transportation Managers were more likely to Disagree (38 percent) than to agree (33 percent) with the statement that their counterpart resource organizations cared about their own agency's mission (item 2 above). Well under half of the Reviewers (39 percent of transportation, 44 percent of resource) agreed with this statement.

Fewer than half of the transportation officials agreed that there was a sufficient level of trust between their agencies and their resource counterparts (item 5 above). Just over half of the resource officials agreed with this statement. Nearly identical response patterns were observed for item 6, focusing on the commitment of the counterpart organization to making the environmental review process run efficiently, and item 7, about whether the counterpart organizations is willing to compromise. On the latter item, more transportation Managers expressed Disagreement (35 percent) than agreement (28 percent), similar to the pattern observed on item 2 above.

Transportation officials, especially Managers, gave significantly more negative responses than did resource officials to the two parallel questions about commitment to their own agency's priorities. Only 26 percent of transportation Managers agreed that their counterpart resource organizations were committed to transportation improvements, while 44 percent Disagreed. Transportation Reviewers were equally divided between agreement, Disagreement, and the neutral (neither agree nor Disagree) response.

In contrast, nearly half of resource Managers (49 percent) agreed that their transportation counterparts were committed to protecting the environment and far fewer (19 percent) Disagreed. The corresponding figures for resource Reviewers were 44 percent and 24 percent.

Characterization of the Quality of Interagency Relationships

Finally, all four groups of officials were asked to characterize the quality of the overall relationship with their named counterpart organizations on a 5-point scale (where 1 = "poor" and 5 = "excellent"). With the exception of transportation Managers, the officials all gave similar responses: 61 to 63 percent giving scores of "very good" or "excellent," 30 to 32 percent giving a neutral score of "3," and only 1 to 3 percent giving scores of "poor." However, only 47 percent of transportation Managers indicated scores of "4" or "5" on this item, repeating a pattern found on the more specific evaluation items.

Most officials involved in the environmental streamlining process expressed belief that their agency's relationship with their counterpart organizations have improved over the last three years (ranging from 49 percent for transportation Reviewers to 59 percent for resource Reviewers. However, just over one-third of the officials (33 to 38 percent) indicated that the relationship had not changed over the past three years. A small but significant minority (14 to 16 percent of transportation officials, and only 7 percent of resource officials) reported believing that the relationships had gotten worse over the past three years.

Overview

Environmentally responsible transportation improvements, delivered on time and within budget, is a simple vision that all too often evades the U.S. Department of Transportation (DOT) and its partner agencies. Public expectations and demands for effective transportation solutions on a reasonable time frame are understandable, given the magnitude and pervasiveness of America's highway transportation problems. Equally understandable is the public's desire for environmentally sound ways of providing for sustainable transport solutions. Inevitably, these two societal goals occasionally come into conflict.

The term "Environmental Streamlining" describes the effort to combine the timely delivery of transportation projects with the protection and enhancement of the environment. First enacted into legislation for highway and transit projects with the Transportation Equity Act for the 21st Century (TEA-21), Environmental Streamlining requires transportation and environmental agencies to cooperate in developing realistic project time frames, and then work collaboratively to adhere to that schedule. Because major transportation projects are affected by dozens of Federal, State, and local environmental requirements administered by a multitude of agencies, improved inter-agency cooperation is critical to the success of environmental streamlining.

Efforts currently underway within the Federal Highway Administration (FHWA) of the U.S. DOT focus on solidifying the inter-agency partnership through a series of actions that include pilot efforts, process reinvention, alternative dispute resolution, and a focus on process and outcome evaluation. Experience in implementing environmental streamlining will lead to critical policy choices that may point to the need for revisions to transportation or environmental laws or regulations.

The objectives of the environmental streamlining provisions, as described in the explanatory materials on TEA-21, prepared by the Committee on Transportation and Infrastructure, are as follows:

• to establish an integrated review and permit process that identifies key decision points and potential conflicts as early as possible,
• to integrate the National Environmental Protection Act (NEPA) process into each project as early as possible,
• to encourage full and early participation by all relevant agencies that must review a highway construction project or issue a permit, license, approval, or opinion relating to the project, and
• to establish coordinated time schedules for agencies to act on a project.

As part of its response to the charge of Section 1309 for streamlining environmental reviews of proposed transportation projects, FHWA is investigating both qualitative and quantitative data sources aimed at creating sound, actionable performance measures for the project development process. These include:

1. A study focusing on establishing an historic baseline for assessing time frames for project development is currently underway. The goal of this research is to establish a baseline against which to assess project delays over time; to attempt to identify causes for such delays (reviews, applications for permits, etc.); and to assess through quantitative statistical analyses the trends and determinants of delays. The baseline study is examining those transportation projects for which Environmental Impact Statements (EIS) were prepared.
2. FHWA, in conjunction with the Center for Transportation and the Environment (CTE), has proposed a series of informal interviews with Federal agencies as well as a limited number of non-Federal agencies/entities for the purpose of assessing the perspectives of and attitudes towards environmental streamlining that are held by those involved in the delivery and environmental review of transportation projects. The interviews will be designed to gather background information on the target stakeholders' perceived roles, expectations, problems, and obstacles (e.g., differing interpretation of rules) with regard to streamlining the project development process. This background information is expected to contribute to the development of performance measures and indicators to assess the effectiveness of environmental streamlining.
3. The Transportation Committee of the American Consulting Engineers Council (ACEC) proposed conducting a survey to measure the performance of stakeholders in the transportation project development process (i.e., resource and transportation agencies). This study would measure perceptions of key participants in the project development process, and, by means of applying scientifically reliable and valid survey methods, explore how stakeholders in the process view the quality of the environmental work and services performed by their counterparts. The survey will include questions referring to:
   • Whether staff members of resource agencies believe that State Transportation agencies are submitting thorough documentation in fulfillment of permit application requirements
   • Whether staff members working in State Departments of Transportation (DOT) perceive that resource agencies are providing timely and constructive responses to the permit process.
   Findings from this survey will enable both parties identify potential problems and to improve inter-agency communication throughout the process.

Research Objectives and Study Design

The Gallup Organization was commissioned to design and conduct the third study listed above with the objectives of:

1. Measuring the performance of agencies involved in environmental streamlining in order to provide a benchmark for agencies to gauge their own performance and that of the process itself.
2. Identifying aspects of the process that represent opportunities for improvement.

Questionnaire Development

Gallup's research team developed initial drafts of both the resource and transportation agency questionnaires. The project began in the second half of 2001, with Gallup conducting qualitative focus group sessions in Chicago, Denver, Atlanta, Boston, Dallas, and San Francisco¹ . Using the results received from the qualitative findings, a draft survey questionnaire was developed and tested in cognitive process interviews conducted through a second series of focus groups. This approach permitted evaluation of multiple versions of the survey questions before finalizing the instrument.

¹ FHWA: Environmental Streamlining and the Development of Performance Measures: A Detailed Analysis of Focus Group Findings. The Gallup Organization, January 11, 2002.

Responses from focus group participants revealed common themes concerning relationships among the counterpart transportation and resource-permitting agencies. For that reason, two sets of questions were developed:

• items focusing on broad aspects of the relationship between the agencies that develop and extend over considerable time, and
• items focusing on specific interactions on a particular project, emphasizing activities over which the agencies are believed to have considerable management discretion and control.

Resource Agency and Transportation Agency Questionnaires

Two questionnaires were designed for use in this study, one intended for transportation officials and the other for resource agency officials. The questionnaires were almost identical in their layout.

Both questionnaires contained four sections organized around the following topics:

1. 16 screener questions to identify eligible respondents, where eligibility is defined as having recent experience reviewing and acting on NEPA documents or managing the review process for NEPA projects,
2. 36 project-specific ratings on several dimensions of the quality of the process and the performance of counterpart agencies (reviewers in resource-permitting agencies rated the performance of transportation agencies and vice versa). These included a rating of the respondent's overall satisfaction with the project, 4 questions on problem incidence, 7 general relationship evaluation questions, 10 communication questions, 5 timeliness questions, and 9 general project performance questions.
3. 11 overall agency ratings not focusing on any specific NEPA project, including 9 questions about the interagency relationship and two questions about the agency's general performance.
4. 4 demographic questions about the respondent.

Using questionnaires developed from focus group findings, a pilot study was conducted in DOT Region 5 to assess both the instruments and procedures for identifying qualified respondents.² Based on findings from the pilot test, the questionnaires were modified slightly prior to conducting the full-scale survey in the remaining nine DOT regions using Computer Assisted Telephone Interviewing (CATI) technology.

To accommodate respondents who wished to respond to the survey but were not able to schedule a telephone interview during the survey period, the CATI questionnaires were adapted for self-administration using an electronic survey form on the World Wide Web (the CATI and Web questionnaires were identical in content). Please contact Kreig (Chip) Larson at 202-494-2056 or kreig.larson@fhwa.dot.gov to receive a copy of these questionnaires.

² FHWA: Environmental Streamlining and the Development of Performance Measures: FHWA Study - Pilot Study Results. The Gallup Organization, October, 2002.

Specifying the Population of Interest

Identifying Eligible Respondents in Transportation and Resource Agencies

Because no list of environmental streamlining participants existed, the list of potential survey respondents was constructed using several methods. First, FHWA and Gallup assembled a comprehensive inventory of agencies and organizations that are routinely involved in environmental streamlining activities on transportation projects. Gallup staff then located telephone numbers from a variety of information resources, and called the organizations to obtain the names of managers and reviewers involved in transportation projects. Members from the National Technical Review Board also supplied names and telephone numbers for likely sample members. Following quality control checks for accuracy, the initial list of names and telephone numbers was entered into the CATI scheduling and management system and the survey was initiated.

The initial segment of each survey call consisted of a series of screening questions designed:

• to confirm that staff of the contacted organization had been directly involved in the environmental streamlining process during the past six months, and especially within the last 90 days,
• to determine whether individuals being called were eligible for the survey by virtue of their specific roles in the environmental streamlining process, and
• to use the initial contacts at the organization as informants to identify all other persons in the same organization who were also eligible to be interviewed as a result of their roles and duties on transportation projects that included environmental streamlining efforts.

The list of agencies contacted for the transportation survey included:

• Federal Highway Administration
• Federal Transit Authority
• State Departments of Transportation
• Federal Aviation Administration
• Mass Transit Agencies
• City and/or County Transportation Authorities

The list of agencies contacted for the resource survey included:

• National Park Service
• National Forest Service
• Fish and Wildlife Service
• Bureau of Land Management
• National Marine Fisheries
• Environmental Protection Agency
• U.S. Army Corps of Engineers
• Federal Lands Program
• U.S. Coast Guard
• State Department of Natural Resources/Department of Environment
• State Historic Preservation Organizations

Respondents who completed the interview were asked if they could identify others in their agency performing similar work who would thus qualify to be interviewed. If so, additional names and telephone numbers were recorded and these potential respondents were called to confirm eligibility and conduct the interview. Using this "networking" approach, we were able to identify the vast majority of managers, program reviewers, and other technical staff who had direct experience with transportation projects during the six-month period preceding the interview date.

Data Collection

The CATI survey was conducted from Gallup's Executive Telephone Interviewing Center in Lincoln, Nebraska. The self-administered electronic questionnaire was programmed and operated on Gallup's secure Web servers.

Selection and Training of Telephone Interviewers

Five Executive Interviewers were selected based upon their prior experience interviewing similar types of professionals using similar screening methods and survey questions. Gallup survey managers produced a training manual to provide project-specific guidelines for the screening, interviewing, and "networking" procedures to identify additional eligible respondents.

Training sessions were conducted in Lincoln, Nebraska, with a videoconference from Washington, D.C. attended by Gallup and FHWA project staff for the pilot study. Interviewers were trained for three hours on the use of the FHWA questionnaires, techniques for understanding screening procedures, and a review of standard responses to commonly asked questions associated with the questionnaires. The training materials included a glossary of acronyms for agencies that interviewers would contact, and detailed specifications for each of the survey questions.

Survey Notification Letters

Prior to contact by telephone, all prospective survey respondents were sent notification letters signed by the FHWA Administrator explaining the purpose of the survey and asking for their cooperation. A copy of the Administrator's letter is attached to this report.

CATI Interviews

Telephone interviewing began on October 28, 2002, and continued through April 18, 2003 for officials in transportation agencies and through May 16, 2003, for those in resource organizations. Interviewing was temporarily suspended from November 1, 2002, through February 1, 2003, in order to obtain an extension of the expiration date for the forms clearance issued by the Office of Management and Budget.

Web-based Self-Administered Interviews

To maximize response rates, a Web-based, self-administered electronic form was developed to provide respondents with an alternative mode to telephone interviewing. The Web-based form was made available to respondents from November 10, 2002, through the conclusion of the CATI survey period. In total, 92 individuals completed the self-administered questionnaire rather than the CATI interview. The availability of the Web-based survey form increased the overall response rate by four percentage points (from 58 percent to 63 percent).

Web responses by region were examined, and in only one region, Region 10, did the number of Web responses go above 15% of total responses (Web responses in Region 10 were 20% of the total transportation sample). However, in the data, Region 10 does not show up as a region with extensive difficulties, and for that reason the modal effects are not considered a hindrance to the results. It is something that needs to be carefully considered in the future and among future survey work done for this group of respondents.

Follow-Up Procedures

As many as seven calls were made on different days of the week and during different hours of the day in order to contact each potential respondent. Once contact was made, as many as seven follow-up calls were made in an attempt to complete an interview with that person. When an answering machine or voice mail was encountered, interviewers left a message that included a toll-free telephone number encouraging individuals to call Gallup at any time they found convenient. Numbers found to be non-working or otherwise incorrect were checked for accuracy and updated by contacting the resource organization directly.

Interviewers attempted to contact a total of 1,596 officials in resource organizations and 1,171 officials in transportation agencies. The table below shows the final disposition of all cases included in the final list of potential respondents. A total of 649 of the officials contacted were ineligible to be surveyed (80 in transportation agencies and 583 in resource organizations). Survey data were obtained from a total of 709 transportation officials and 608 resource officials for an overall response rate of 63 percent (65 percent among transportation officials and 60 percent among resource officials).

These completion rates are based on the most conservative assumptions about the eligibility — namely that all the individuals who were not contacted would have been eligible to be interviewed. However, among the cases that were contacted, we determined that 80 of 1,171 transportation officials we contacted and screened were ineligible (7.0 percent), and 583 of the 1,596 resource officials we screened were also ineligible (37.0 percent). Applying these ineligibility rates to the non-contacted cases, the adjusted response rates would be 66 percent for transportation officials and 70 percent for resource officials, and an overall adjusted rate of 68 percent.

Only 47 of the nearly 2,800 individuals we attempted to contact refused to participate in this voluntary survey.

Final Completion Rates

	Transportation	Resource	Total	Combined % of Total
Total Contact/Screening Attempts	1,171	1,596	2,767
Screen Failure/Ineligible	62	73	125
Screen Failure/Don't Know-Refuse	18	29	43
Ineligible (due to responsibility level)	0	481
Total Potentially Eligible	1,091	1,013	2,104	100%
Complete CATI Interview	630	595	1,225	58%
Completed Web Questionnaire	79	13	92	4%
Total Completed Cases	709	608	1,317	63%
Completion Rate	65%	60%	63%
Non respondents
Call Back/Answering Machine/No Answer	180	329	509	24%
Other	124	20	144	7%
Refusal	15	32	47	2%
Disconnected Numbers	10	11	21	1%
Total Nonrespondents	329	392	721	34%

The findings presented in this report are based on the following numbers of completed interviews in each DOT Region:

The remainder of this report presents the findings of the full-scale survey in nine regions, integrating data from the pilot study when wording and context of the survey items were identical in the pilot and full-scale surveys. These findings represent baseline data that we believe may be used to develop an ongoing performance measurement system on the environmental streamlining process. Subsequent data collection efforts will be able to compare changes in performance measures resulting from a variety of process improvement efforts.

The 2003 FHWA Survey of Transportation Agencies and Resource-Permitting Organizations

The following section provides an overview of the survey respondents who participated in the 2003 survey.

Whom did we interview?

As shown in Exhibit 1, interviews were conducted with a total of 709 individuals working in transportation agencies and 608 persons working in resource-permitting organizations. Within each type of agency or organization, roughly half of the interviews were conducted with people who conduct reviews or work on obtaining NEPA document approvals on a day-to-day basis (referred to as "Reviewers" in this report). The remaining interviews were conducted with individuals who have upper-level management responsibility for NEPA projects and are not typically in projects on a daily or frequent basis, yet have considerable knowledge about the relationship between their agency or organization and their counterparts (referred to as "Managers"). As shown in Exhibit 1, by a small margin, the majority of interviews in transportation agencies were conducted with Reviewers, whereas the slight majority of interviews in resource organizations were conducted with Managers. The numbers of interviews completed in each type of organization are more than adequate to carry out the planned analyses of the survey data on a national level. When the same questions were asked of both Reviewers and Managers, the numbers of observations are also sufficient to examine results within the 10 DOT Regions.

Back to Table of Contents

Respondents from transportation agencies had considerable seniority in their agencies and experience in NEPA-related responsibilities. Those identified as day-to-day Reviewers reported working an average of 13.1 years in their agencies and 12.0 years on NEPA-related projects. Managers in transportation agencies had an average of nearly 17.7 years tenure in their agencies and 14.6 years with NEPA projects.

Respondents from resource organizations reported similar levels of seniority. Day-to-day Reviewers reported an average of 13.3 years working in their organizations and an average of 12.5 years working on NEPA-related projects. Managers in resource organizations reported working in their agencies for 16.5 years, with an average of 15.2 years experience working on NEPA projects.

Male/female ratios were generally consistent in the two types of organizations. Among respondents from transportation agencies, 68 percent of the Reviewers interviewed were male and 32 percent were female, while 81 percent of the Managers were male and 19 percent were female. In resource or permitting organizations, 67 percent of the Reviewers interviewed were male and 33 percent were female, while 70 percent of the Managers were male and 30 percent were female.

Respondents served in a wide variety of professional positions. For the resource agencies, these included:

• Archaeologist
• Biologist
• Chief Environmental Officer
• Design Engineer
• Director of Planning
• District Engineer
• Division Chief
• Engineering Manager/Supervisor
• Environmental Assessment Manager
• Environmental Coordinator/Program Manager/Specialist/Analyst
• Environmental Manager/Supervisor
• Environmental Policy
• Environmental/Civil Engineer
• Fish and Wildlife Biologist
• Landscape Architect
• Preservation Specialist
• Transportation Planner/Specialist

For transportation, the professions of respondents included:

• Chief Environmental Officer
• Design Engineer
• Director of Planning
• District Engineer
• Division Chief
• Engineer/Senior Engineer
• Environmental Policy/Planner/Coordinator/Specialist/Analyst/Program Manager
• Environmental Engineer/Manager/Supervisor
• Transportation Specialist
• Archaeologist
• Environmental Planner
• Biologist
• Civil Engineer
• Operations Engineer
• Transportation Planner/Engineer

Exhibit 2 shows that the great majority of Reviewers in both types of organization had recent experience working on a NEPA project: 97 percent of respondents in transportation agencies and 87 percent of respondents in resource organizations had completed work on a NEPA project within the last 90 days. Moreover, all remaining Reviewers had direct involvement in a NEPA project within six months prior to the date of interview (but had not completed a project in the last 90 days). Thus, all Reviewers were able to provide judgments about relevant and recent performance by both their own organization and their counterparts.

Back to Table of Contents

Reviewers in both types of organizations had roughly similar levels of experience with NEPA projects in the 90 days prior to the interview. Reviewers in transportation agencies reported working on an average of 11.2 Federally funded permits or approvals. Reviewers in resource organizations reported working on an average of 14.5 reviews, approvals, or permits in the 90 days prior to the interview. (Reviewers in resource organizations were not restricted to only Federally-funded projects, consequently they reported more projects.) Reviewers in both types of organization were asked to count only those projects that were completed and to ignore projects that were still in process. Managers were not asked about the number of projects they oversaw in the 90 days prior to their interview.

What Types of NEPA-Related Projects Were Evaluated by Reviewers?

Reviewers in transportation agencies were asked to name the most recent Federally-funded NEPA-related project they had worked that required the most interaction on their part with staffs of resource or permitting agencies. By identifying projects with high levels of interaction with counterpart agencies, we sought to ensure that the performance ratings given by each reviewer would focus on a recently completed project with a sufficient frequency of interaction with resource organizations to provide meaningful data on project processes, problems, and organizational interactions.

Similarly, reviewers in resource-permitting organizations were asked to name the recent NEPA-related project they had worked that required the most interaction on their part with staffs of transportation agencies. As noted above, Reviewers in resource organizations were not asked to restrict their attention to only Federally funded projects in identifying the project to be evaluated. Our purpose was primarily to identify the project that involved the greatest quantity of interaction with their transportation agency counterparts at the Federal or State levels.³

³ If respondents named more than one NEPA project as having similarly high levels of interaction, a random selection of one project was made using a function of the interviewing software.

Once a single high-interaction project was identified, Reviewers in both types of organization were then asked to classify the project using the categories shown in Exhibit 3:

• EIS (Environmental Impact Statement)
• EA FONSI (Environmental Assessment — Finding of No Significant Impact)
• FHWA CE (Federal Highway Administration — Categorical Exclusion)
  or
• Programmatic Agreements

Over 99 percent of the transportation Reviewers and over 93 percent of the resource organization Reviewers were able to classify their selected projects using one of the categories provided.

The distributions of project types differed somewhat between transportation and resource Reviewers. For transportation Reviewers, there was a roughly even distribution across EIS (27 percent), EA FONSI (35 percent), and CE projects (32 percent), with only 5 percent focusing on Programmatic Agreements as the projects with the most interaction with resource or permitting organizations. Reviewers in resource organizations were about evenly divided in specifying EIS projects (41 percent) and EA FONSI projects (39 percent), with only 8 percent naming CE projects and 12 percent identifying Programmatic Agreements as the projects with the greatest interaction with transportation agencies. In our view, the differences in distributions across project types are not large enough to affect the distribution of attitudes and ratings by Reviewers from either type of organization.

Back to Table of Contents

Which Transportation Agencies and Resource-Permitting Organizations Did Reviewers Evaluate?

Transportation and resource Reviewers were also asked which counterpart agency or organization they dealt with most on the project they named. Reviewers in transportation agencies were most likely to report interacting with the Army Corps of Engineers or ACOE (27 percent), the US Fish and Wildlife Service or US FWS (18 percent), and State Historic Preservation Offices or SHPO (14 percent). These three agencies were identified as the primary counterpart organizations by about 60 percent of the transportation Reviewers. In response to a parallel question, over 85 percent of resource Reviewers named a State (81 percent) or local (4 percent) Department of Transportation or DOT. Most of the remainder named a Federal agency such as FHWA, ACOE, FAA, or Coast Guard, with FHWA cited by 11 percent.

Managers in transportation agencies were asked to name the resource organizations (not a specific project) that their staffs interacted with most on NEPA-related projects. Altogether, about 60 different resource organizations were named by the 318 transportation Managers. Of these, the most commonly cited were the ACOE by 24 percent, the US FWS by 20 percent, SHPO by 9 percent, and the US Environmental Protection Agency (EPA) by 9 percent. Together, these four agencies account for over 60 percent of the citations by transportation Managers.

Similarly, Managers in resource associations were asked to name the transportation agencies their staffs interacted with most on NEPA projects. The 326 resource Managers listed about 40 different transportation organizations. The most commonly mentioned agencies were State or county DOTs (by 59 percent, in 35 States), and FHWA by 21 percent. Together, the State and local DOTs and FHWA account for over 80 percent of the agencies cited by resource Managers.

Report of Major Findings

A Note on Interpreting the 2003 NEPA Process Survey Results

In the following sections, we analyze the survey data on perceptions and experiences of Reviewers and Managers working in transportation agencies and resource or permitting organizations. Reviewers in both types of organization were asked for their evaluations of their experiences on the NEPA project they cited in the screening section of the questionnaire (the project that required the most interaction with their counterpart organizations on the other side), and with the organization or agency they identified as the one they interacted with most. Because both types of Reviewers named a large number of different projects and worked with several different counterpart agencies or organizations, we will use the phrases "cited project" and "named organization" to refer to the diverse list of projects and organizations identified in the screening section of the interview. These two phrases are intended to remind the reader that all of the perceptions and beliefs by transportation and resource officials about the NEPA process and performance attributes of involved organizations must be understood as referring to a self-selected sample of projects and interaction patterns.

Readers should bear in mind that participating Reviewers were permitted and encouraged to identify the single project in recent experience that had the highest level of interactions between them and their counterpart agencies on the other side of the process. No constraints requiring maintained focus on the same project were placed on transportation and resource Reviewers — even those located in the same DOT regions. Not surprisingly, a large number of different projects were cited by the 391 transportation agency Reviewers and the 282-resource organization Reviewers.

The data suggest that in a few instances, some of the Reviewers from both transportation agencies and resource organizations in the same DOT region may have provided ratings on their counterparts with whom they interacted on the same project.⁴ In those very few cases, the survey data from transportation and resource Reviewers contain responses in which the reviewers are actually rating organizations and possibly individuals who are also respondents to this survey. However, the frequency that transportation and resource Reviewers focused on a common project and on each others' organizations was far too low to allow this survey to be used to evaluate either the process or agency performance on specific NEPA projects. This survey was not designed for that purpose and cannot be validly used for that purpose.

⁴ Because the survey did not ask respondents to provide the formal name and identifying number of their selected project, the survey data cannot be used to identify specific NEPA projects with certainty. The absence of formal project designations is a design feature of the survey that was intended to enhance respondents' privacy and confidentiality, and hence the validity and accuracy of the performance evaluations they provided.

The analyses and descriptions in the following sections represent characterizations of general project and agency performance attributes by the Reviewers and Managers who were interviewed. They cannot and should not be interpreted as evaluations of particular projects in any DOT region. For this reason, our emphasis in what follows is on the evaluation of the process, problems, perceived agency attributes, and relationships between the two types of agencies all across the country and, in some cases, within the ten DOT regions. With project-level ratings from almost 700 Reviewers and almost 650 managers of NEPA-related activities across the country, these data provide a unique opportunity to understand what the participants in the environmental streamlining process believe to be both the major strengths and the most important opportunities for improvement for both the NEPA process and the relationships among the organizations that must interact to complete the projects. By understanding these general characteristics of the process and the interactions and relationships among participating organizations, senior managers representing both sides of the NEPA process can identify or develop adjustments that have the greatest potential for improving the effectiveness and efficiency of agency collaboration and for removing perceived barriers to better, more timely results in NEPA actions.

How Did Transportation and Resource Reviewers Rate Their Experiences on Recent NEPA Projects?

Following the screening questions, the first substantive section of the survey instrument asked Reviewers in transportation agencies and resource organizations (those with day-to-day responsibility for working directly on NEPA projects), a set of questions about their experiences on the cited project and with the named organization recorded in the screener. Managers in transportation agencies and resource organizations were not asked project-level questions but were skipped to a set of questions about their perceptions of organizational relationships (that were also asked of reviewers). Organizational relationship data collected from Reviewers and Managers are analyzed in a later section of this report.

Reviewers Rate Cited Projects and Named Organizations

Reviewers in both types of organization were told:

"Now I'm going to ask you a series of questions about the [cited project] with regard to [named organization]. Overall, how satisfied were you with [named organization]'s performance on this project? Please use a scale from one to five, where 1 is 'very satisfied,' and 5 is 'very unsatisfied.'"

Charts 1 and 2 show the response distributions obtained from transportation and resource Reviewers.

Chart 1. Overall Satisfaction Expressed by Transportation Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

Chart 2. Overall Satisfaction Expressed by Resource Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

On balance, Reviewers from both types of organizations report moderately high satisfaction in their interactions with counterpart agencies on recent projects. However, clearly there is room for process improvements that could further increase the satisfaction of these key officials.

About one-quarter (26 percent) of the 390 transportation agency Reviewers indicated that they were "very satisfied" with the named resource organization's performance on the cited project, while only one in twenty (5 percent) said that they were "very unsatisfied" — over a 5 to 1 ratio of high satisfaction to deep dissatisfaction. One in five (20 percent) of transportation Reviewers selected the neutral response (a value of 3 on the five-point scale) indicating neither satisfaction nor dissatisfaction. However, the remaining 50 percent of the transportation Reviewers were far more likely to indicate they were satisfied with the process (41 percent) than dissatisfied (7 percent) — nearly a 6 to 1 ratio in positive to negative ratings.

In total, two-thirds (67 percent) of the transportation Reviewers reported being either satisfied or very satisfied with their named resource organization's performance on their cited projects, while less than one-eighth (12 percent) reported being either dissatisfied or very dissatisfied.

A strikingly similar pattern is evident in the responses from resource Reviewers, with over one-fourth (28 percent) indicating they were "very satisfied" with the named transportation agency's performance on their cited projects, and fewer than one in thirty (3 percent) reporting that they were "very dissatisfied." Also similarly, 20 percent of the resource Reviewers chose the middle response category. Among the remaining 49 percent, nearly three times as many resource Reviewers (36 percent) reported they were "satisfied" with the performance of their counterparts, in contrast to only 13 percent who indicated they were "unsatisfied."

In summary, nearly two-thirds of resource Reviewers (64 percent) said they were either satisfied or very satisfied with the named transportation agency's performance, while less than one-sixth (16 percent) responded that they were dissatisfied or very dissatisfied.

The relative symmetry in the distributions of overall satisfaction scores from Reviewers in both types of organization supports the conclusion that in the majority of recent projects, interactions with counterpart agencies are seen as satisfactory. The sizeable percentages of both types of Reviewers reporting satisfaction also support the conclusion that positive ratings by one type do not tend to be balanced by negative ratings by counterparts. We cannot examine this possibility directly because Reviewers in transportation and resource organizations cannot be linked to a specific common project. However, by examining these results at the regional level, we may obtain a somewhat closer

Regional Results in Reviewers' Overall Satisfaction Ratings of Counterpart Organizations

The percentage distributions across the five categories of the overall satisfaction measure varied somewhat by region, but not substantially. Analysis of regional distributions of reviewers is hampered by the total sample size of 390 transportation Reviewers, leaving an average of only 39 observations per region (ranging between 23 Reviewers in Region 1 and 60 Reviewers in Region 9). The regional numbers are even smaller for the resource Reviewers, ranging from only 11 observations in Region 2 to 37 cases in Region 9. Given these small numbers, we examined the percentages in the two "satisfaction" categories combined, and the two "dissatisfaction" categories combined. At this level of aggregation, regional differences were relatively minor.

Across regions, 12 percent of transportation Reviewers reported they were "unsatisfied" or "very unsatisfied" with the performance of the named resource organization. This percentage varied by a maximum of only +/- 6 percentage points across all 10 regions, from 6 percent in Region 6 to 18 percent in Region 9. The combined dissatisfaction levels for most of the DOT Regions were within 4 percentage points of the overall percentage (12 percent), supporting the conclusion that dissatisfaction with NEPA project interactions is at a relatively low level in all regions.

Slightly greater variation was found on the satisfaction side of the scale, with Region 3 (at 54 percent) being 13 percentage points below the overall average of 67 percent, and Region 5 (at 81 percent) being 14 percentage points above the overall average. However, most of the DOT regions were within the range of 4 percentage points below and 7 percentage points above the overall average.

For the smaller number (282) of resource Reviewers, regional variation in satisfaction distributions appeared slightly larger than for transportation Reviewers, but this is primarily a consequence of the smaller numbers of resource Reviewers in all regions. Distributions that deviated most from the overall results arose because of very small cell sizes (zero to 3 cases) occurring in Regions with very few respondents. Despite the smaller cell sizes, the regional distributions of satisfaction for resource Reviewers were sometimes similar and sometimes different from those of transportation Reviewers. Regions 2, 9, and 10 showed somewhat higher levels of dissatisfaction, while Regions 3, 4, and 7 reported lower levels of dissatisfaction compared to the overall average for resource Reviewers. Conversely, resource Reviewers in Regions 1, 3, and 7 reported higher levels of satisfaction, and those in Regions 2, 8, and 10 indicated lower percentages of satisfaction than the overall average for resource Reviewers.

A comparison of mean (average) satisfaction scores calculated for the two types of Reviewers in the 10 DOT regions is displayed in Exhibit 4 as follows.

Back to Table of Contents

Across regions, the difference in mean satisfaction score between transportation and resource Reviewers is extremely small (+0.03), and is of no relevance or significance for policy or management purposes. Within regions, the differences are somewhat larger. In half of the Regions, the transportation Reviewers report slightly higher satisfaction scores, and the opposite is true in the other five. All ten of the Regional differences fall within the range of -0.3 to +0.4 points on the five-point satisfaction scale.

An intuitive understanding of the magnitude of these differences may best be gained with an example: To raise the mean score of all Reviewers from the current level of about 3.74 to a 4.00 would require just over one-fourth of the Reviewers (175 individuals) to increase their satisfaction rating by 1 point on the five-point scale over their 2003 rating (e.g., from "dissatisfied" to "neutral," or from "neutral" to "satisfied"). In the 2003 survey, a total of 230 Reviewers (34 percent) reported being less than "satisfied" (values of 1, 2, or 3 on the scale); from that number, 138 reported "neutral" scores (3).

Because of the inherent unreliability and wide swings of percentage distributions based on small numbers of observations, we will restrict most of our analyses and interpretation of Reviewer responses to project-specific questions to the overall sample.

Implications for Management

Overall satisfaction with the performance of counterpart agencies is an important indicator of the efficiency and effectiveness of NEPA projects. Under ideal circumstances, all Reviewers from both types of organizations would report they were "very satisfied" with their counterparts' performance on projects that required the highest levels of agency interaction. However, our survey data indicate that about three-fourths are less than "very satisfied" with their counterpart agency's performance, leaving a great deal of room for "improvement" in the relationships and interactions.

That said, given the partly adversarial nature of the NEPA process, universal reports of the highest level of satisfaction among reviewers is an unrealistic expectation. Moreover, a survey measurement approach that asks Reviewers about the project that required the most interaction with a counterpart organization unavoidably results in selection of projects likely to have more contentious elements than the average NEPA project. An alternative goal of all reviewers reporting that they are either satisfied or very satisfied would be a strongly positive outcome for the environmental streamlining effort. At this stage, for both transportation and resource Reviewers, one-third or slightly more did not report that level of satisfaction on their cited projects. The goal of having no Reviewers report overall satisfaction scores of 3 or lower may also be practically unachievable, given the nature of NEPA projects.

The mean (average) satisfaction scale scores given to their counterparts by both transportation and resource Reviewers were nearly identical at values of 3.76 and 3.73, respectively, on the five-point scale. By definition, a mean value of 5 on the five-point scale would indicate that all reviewers reported that they were "very satisfied" with their counterpart's performance on the cited project — again, an unrealistic standard. However, a mean value of, 4.5 on the five-point scale would be a meaningful target, as it would represent a high level of average overall satisfaction by Reviewers with their counterpart organizations, but would not require the complete elimination of scores of 3 or lower. The current mean satisfaction level for Reviewers in both types of organizations is at about 83 percent of a hypothetical target mean value of 4.5. Interventions that would raise the performance evaluations to a mean of 4.5 within a one- to two-year period may be developed using information from the 2003 survey on those areas of project performance that are most frequently cited as problem areas by both transportation and resource Reviewers.

Raising overall average satisfaction scores to 4.5 is a realistic goal for the next few years. A useful management strategy will be to determine the types of problems and organizational relationship and performance attributes most associated with reviewers assigning scores of 3 or lower on the overall satisfaction measure, so that interventions can be designed to reduce the occurrence of those problems, organizational characteristics or behaviors, and other aspects of NEPA project interactions. Even more importantly, senior Managers should identify areas of excellent performance as identified by Reviewers and Managers in both types of organization across the ten DOT regions, and conduct further research to determine whether the responsible staff and organizations in those Regions have developed or identified solutions to problems that may be observed in other Regions. All organizations gain in effectiveness and efficiency when the methods and procedures found in the best performing segments of the organization are identified and diffused throughout the rest of the organization.

Reviewers' Experience of Problems with Their Counterpart Organizations on NEPA Projects

Following the overall satisfaction question, Reviewers for both types of organizations were asked:

"Did you experience any problems with [named organization] during the [cited project]?"

Just over one-third (36 percent) of transportation agency Reviewers reported that they experienced problems with the named organizations on their cited projects, while nearly two-thirds (64 percent) indicated that they had not encountered problems.

Similarly, 37 percent of resource Reviewers indicated that they had experienced problems dealing with their counterpart agencies on their cited projects, and 63 percent reported no problems.

Because these distributions are very similar to those for the overall satisfaction question reported just above, we examined the extent of the relationship between the satisfaction score given and the reports of project problems. As expected, the correlations between the responses were very high, but they were not perfect, so the two questions are not measuring precisely the same reactions. Not surprisingly, the transportation and resource Reviewers who reported experiencing project problems were much more likely to also report being "dissatisfied" or "very dissatisfied" with the performance of the named organization on the cited project. Fully 34 percent of transportation Reviewers who reported problems with their resource counterparts also reported being dissatisfied with their performance, and another 36 percent gave a neutral response to the satisfaction question. Among resource Reviewers, 40 percent of those who reported problems with their transportation counterparts also reported being dissatisfied or very dissatisfied. An additional 31 percent of resource Reviewers who reported problems gave a neutral response to the satisfaction question.

We note that about 30 percent of both types of Reviewers who reported having project problems also indicated they were "satisfied" or "very satisfied" with their counterpart organizations.

For both transportation and resource Reviewers who did not report problems, less than 1 percent indicated that they were dissatisfied, and only 12 to 14 percent gave neutral responses (lower than the 20 percent giving neutral scores overall).

We believe at least two types of Reviewer reactions lead to these results. First, some Reviewers (of both types) may have experienced problems with the named organizations on their projects that were so minor that they did not preclude a satisfactory process and/or outcome. Second, based on results of focus group sessions held prior to the survey, we understand that many Reviewers involved in NEPA projects expect to experience problems with their counterpart organizations. Because these problems are anticipated, and may be viewed by some Reviewers as unavoidable and inherent in the process, they are not sufficient to cause expressions of dissatisfaction as measured in the 2003 survey.

Yet, it is clear that the occurrence of project problems with the named organization is strongly associated with overall satisfaction levels reported by Reviewers. Among transportation Reviewers, 87 percent of those who did not experience problems indicated that they were satisfied or very satisfied with their named counterpart organizations. Similarly, among resource Reviewers who did not experience problems, 85 percent indicated they were satisfied or very satisfied with the performance of their named counterpart organizations. It is reasonable to expect that reducing or eliminating problems by examining root causes and assisting transportation and resource organizations to achieve better, faster responses will be the primary key to improving overall performance and the satisfaction ratings given in response to evaluation survey questions.

Chart 3. Transportation Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

Chart 4. Resource Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

Timing and Nature of Project Problems Reported by Reviewers

To probe the associations between problem occurrence and satisfaction with the named organizations, the 140 transportation Reviewers and the 105 resource Reviewers who reported project problems were asked a series of questions about the nature of the problems they experienced.

First, they were asked, "At what stage of the process did the problem occur?" They then were read the following series of eight stages of project development when problems might be expected to arise:

1. Early project planning or scoping
2. Defining purpose and need
3. Information or data collection
4. Development and analysis of alternatives
5. Analysis of impacts
6. Selection of preferred alternatives
7. Commitment to mitigation measures
8. Finalizing documents or responses to comments

Reviewers were allowed to report problem occurrence at any or all of the eight listed project stages or activities. The following distributions of problem occurrence were reported by transportation and resource Reviewers.

Back to Table of Contents

Transportation and resource Reviewers thus showed some similarities, but also some important differences in their reports on the stages of NEPA projects when problems arose. For both groups of Reviewers, the stage of "defining purpose and need" of the project was the least likely point for perceived problems to occur and the stage of "analysis of impacts" was rated as one of the most frequent problem stages. Nearly two-thirds of the transportation Reviewers who reported project problems indicated that they occurred at both the "analysis of impacts" and the "finalizing documents or responses to comments" stages.

On the resource side, nearly two-thirds of Reviewers cited problems at the "analysis of impacts" stage and the "information or data collection" stage.

Transportation Reviewers were more likely than resource Reviewers to report that problems occurred at the "commitment to mitigation measures" stage.

Resource Reviewers were more likely than transportation Reviewers to report that problems arose at one of the remaining three stages listed, including "early project planning or scoping," "development and analysis of alternatives," and "selection of preferred alternatives."

At least one-third of the Reviewers of one type or the other cited problems in all 8 of the project stages listed in the survey question, indicating that problems occur with significant frequency at all stages of NEPA projects. At least 40 percent of the Reviewers of one type or the other cited problems at 7 of the 8 stages listed. Over half of the Reviewers of one type or the other cited problems at 5 of the 8 listed stages.

More than half of the Reviewers from both types of organizations reported problems with "analysis of impacts" and "finalizing documents or responses to comments," a fact that we believe deserves attention. Beyond these general observations, there are significant differences in the views of the two sides concerning the most problematic stages in NEPA projects.

It is also important to keep in mind, however, that the percentages reported in the prior paragraphs were calculated based on only the 1/3 of transportation and resource Reviewers who reported experiencing any problems in a recent project that required the most interaction with counterpart agencies. In other words, even the most common problems were cited by just over 20 percent of all the Reviewers who were interviewed. Put differently, about 80 percent of the Reviewers on both sides of the NEPA projects did not experience problems with "analysis of impacts." This fact may increase the difficulty of developing effective, targeted interventions that provide universal improvements at this stage of future NEPA projects. That said, data and information collection, impact analysis, and finalizing documents are clearly three areas that may benefit most from management attention on process improvement.

Reviewers were then asked to "please summarize what the (problem was / problems were) in just a few words." Based on earlier focus group sessions, our interviewers were prepared to field-code several "expected" types of responses shown in Exhibit 6 (along with the percentage of transportation and resource Reviewers who named each type):

Back to Table of Contents

For transportation Reviewers, "lack of timely response" was cited more than twice as often as any other issue, and three to five times as often as most of the other types of problems that were mentioned. "Staff turnover or availability" was the second most common type of issue reported by transportation Reviewers, a symptom that could be closely related to the timeliness complaint. Issues such as "problems with mitigation," "poor communications," "wrong or incomplete information," and "Disagreements or differences of opinion" were mentioned by 6 to 9 percent of the transportation Reviewers who reported project problems. In fact, however, these matters were only mentioned by 9 to 12 Reviewers out of the 391 who were interviewed, so they do not appear to us to be very frequent causes of problems.

Among resource Reviewers, there was a more even distribution mentioning each of the listed problems, with "wrong or incomplete information" topping the list, followed by "lack of timely response," "poor communications," "poor coordination," and "problems with alternative interpretation" (cited about equally by 11-12 percent).

In summary, from the transportation side, the single dominant problem associated with dissatisfaction with the performance of counterpart organizations seems to be timeliness of response, which may be related (in the view of transportation Reviewers) to staffing problems at the resource-permitting agencies and may in turn lead to problems with mitigation efforts. From the resource side, project problems that are associated with dissatisfaction with the performance of transportation agencies appear to be a broader mix of process issues led by quality of information provided and timeliness of response, but also including communications, coordination, and other process matters. This understanding provides a framework for the analysis of other ratings collected from transportation agency and resource organization Reviewers, as detailed further below.

Factors Affecting Organizational Interactions on NEPA Projects: The Reviewers' Perspective

Focus groups conducted during the design phase of the evaluation made clear that the nature of organizational interactions on individual NEPA projects are determined by complex combinations of the specific details of each project together with longstanding patterns of agency perceptions and experiences across multiple projects. Depending on their nature, these combinations can produce either of two results: highly efficient processes and successful outcomes, or conflictual interactions that delay project completion and compromise results.

To explore the influence of these factors on their NEPA project experiences, transportation and resource organization Reviewers were asked a series of questions focusing on three aspects of project interactions between the Reviewers' organizations and their named counterpart organizations:

1. The tenor of the relationship between the two organizations on the project (8 questions)
2. The quality and character of communications between the organizations (11 questions)
3. The timeliness of process interactions between the organizations (4 questions)

These questions were asked of all Reviewers,⁵ regardless of whether they indicated in prior survey questions that they'd experienced problems on their cited projects, and regardless of the level of overall satisfaction they'd expressed with their named counterpart organizations' performance.

All 23 items were expressed as positive statements about an aspect of organizational interactions on the cited NEPA project, with which respondents were asked to agree or Disagree on a 5-point scale ranging from 1 (strongly Disagree) through 3 (neutral - neither agree nor Disagree) to 5 (strongly agree). Thus, agreement with the statement implies a positive evaluation of the counterpart organization, while Disagreement represents a negative reaction. Respondents were asked to answer each question in terms of their overall perceptions of their counterpart organization during interactions on their cited NEPA projects.

To emphasize the comparisons of responses between transportation and resource Reviewers, Exhibit 6 displays two rows of percentage distributions for each statement, with percentage distributions for transportation Reviewers in the first row, and those for resource Reviewers in the second row.

We examined Reviewers' responses to these detailed performance evaluation items in an attempt to understand which aspects of agency interactions on the cited NEPA project are most closely associated with the distribution of responses to the overall satisfaction question described above. In addition, we sought to determine whether there were factors that — regardless of their relationship to overall satisfaction with counterpart organizations — would nevertheless be of significant interest to policy makers and senior transportation and resource Managers as opportunities for process improvements on NEPA projects.

Our initial analysis shows that the majority of the 23 detailed measures of Reviewers' perceptions about project interactions are moderately to strongly correlated with responses to the overall satisfaction question. We first review the findings of each group of measures, then describe the results of multiple regression models to determine the relative importance of these attributes to Reviewers' satisfaction levels.

Reviewers' Evaluations of the Impact of Organizational Relationships on the NEPA Project

Six of the items focusing on organizational relationships were asked of both types of Reviewers. The first item is different from the others in that it asks Reviewers to comment on the readiness of their own organizations to participate in the project. Four out of five transportation Reviewers agreed that their agencies knew what was expected of them, with over half indicating strong agreement. Over 90 percent of the resource Reviewers agreed with the statement, with over 60 percent agreeing strongly. Most of the difference between the two types of Reviewers occurs in the middle ("neutral") category. Because the differences are small and the meaning of the neutral category is difficult to interpret, we conclude that Reviewers in the two types of organizations are essentially alike in their perceptions on this issue, but that transportation Reviewers are slightly less likely to feel that the expectations for their agency's role are always clear.

On the remaining five items asked of both types of Reviewers, the response patterns for both groups on three of the items — numbers 2 (providing materials and information), 4 (wanting an active role in the process), and 5 (the agency's opinion counted in the process) — are similar to each other and resemble the distribution for the overall satisfaction measure discussed above, in which 65 to 72 percent expressed agreement (positive evaluations), 12 to 18 percent expressed Disagreement (negative evaluations), and 13 to 24 percent were neutral.

The response distributions were essentially the same for the items that were asked of only one of the two types of Reviewers, numbers 3 (whether transportation agencies appreciated the contribution of resource agencies to the process) and 6 (whether resource agencies regarded the project mission as important).

However, on two of the items about relationships, Exhibit numbers 7 (whether the named counterpart organization "helped to move the project forward") and 8 (whether the named organization "made efforts to improve the process"), significant differences are apparent between the transportation and resource Reviewers. In item 7, 75 percent of resource Reviewers agreed (or strongly agreed) that their named counterpart organization "helped to move the project forward." However, only 56 percent of transportation Reviewers agreed (or strongly agreed) with that statement. Transportation Reviewers were 13 percentage points more likely than resource Reviewers to Disagree with that statement.

A similar pattern is evident in Exhibit item 8, in which 57 percent of resource Reviewers agree (or strongly agree) with the statement that their named counterpart organizations "made efforts to improve the process during this project," compared to only 36 percent of transportation Reviewers. Transportation Reviewers were 14 percentage points more likely than resource Reviewers to Disagree (or strongly Disagree) with the statement.

Primarily for transportation Reviewers, response patterns for these two items are much more negative toward their resource counterparts than the pattern found in both the overall satisfaction measure and the project problem occurrence measure (in which about two-thirds of both types of reviewers gave positive evaluations of their counterparts). When contrasted with the responses of resource Reviewers, the differences in the two distributions appear most striking in the "strongly agree" category — 18 percentage points lower in item 7 and 17 percentage points lower in item 8 — indicating significantly more negative evaluations on these relationship attributes. As we will see further below, these two items reflect persistent differences the views of Reviewers from the two types of agencies — in which transportation Reviewers feel that resource Reviewers are not doing all they might do to reduce the duration of the NEPA projects they cited, and resource Reviewers are more likely to feel that decreasing the duration of their cited projects would have compromised NEPA standards and provisions.

Regional Differences in Transportation Reviewers' Evaluations

When we examined the response patterns for Exhibit items 1 through 8 within each DOT region, we found statistical evidence for regional differences for items 4, 7, and 8 only for transportation Reviewers (response distributions for Reviewers were not significantly different across the 10 Regions).

We noted that transportation Reviewers in Region 9 were well below all other regions in the level of positive evaluations of their resource counterparts about whether they "wanted to play an active role on the project," or "helped move the project forward" (item 7). Because Region 9 has more transportation Reviewer respondents than any other DOT region (59 Reviewers, in contrast with an average of 36 in the other nine regions, and fewer than 30 in three of the Regions), Region 9's distribution had a relatively strong impact on the overall distribution for all ten regions combined. No other Region showed such a substantial departure from the overall pattern for all Regions combined on items 4 and 7.

Regional patterns were much less distinctive for Exhibit item 8. Variations in the pattern of transportation Reviewers responses were distributed in what appeared a random pattern across the fifty cells of the regional breakdown, but no systematic deviation from the overall average was discernable in one or more Regions. We noted that the least positive evaluations of their resource counterparts on "making efforts to improve the process" were indicated by transportation Reviewers in Regions with the fewest respondents, Regions 1 and 7 with 22 and 21 respondents, respectively.

We found no statistically reliable variations across DOT Regions in the patterns of responses given by resource Reviewers to any of the "relationship" items.

Exhibit 7. Reviewers' Perceptions of the Relationships, Communications, and Timeliness of Interactions Between Transportation and Resource-Permitting Organizations

Tenor of the Relationship (items 1 through 8)

Back to Table of Contents

Reviewers' Evaluations of the Impact of Inter-agency Communications on the NEPA Project

Ten of the 11 survey items addressing inter-agency communications were asked of both transportation and resource Reviewers. The eleventh question (item number 9 in the table below) was asked of only resource Reviewers. As shown in the first table below, the response distribution on whether the named transportation counterpart agency involved the resource Reviewer's agency "early in the process" resembles that for the overall satisfaction item. There were no significant variations across regions in resource Reviewers' answers to this item.

The majority of the other items about inter-agency communications also display this common pattern of responses; however, there were notable exceptions:

• Transportation Reviewers were 17 percentage points less likely than resource Reviewers to agree (or strongly agree) that their named counterpart organization "invited our participation in key meetings," with just over half (54 percent) of the transportation Reviewers answering this way.
• Transportation Reviewers were also 17 percentage points less likely than resource Reviewers to agree (or strongly agree) that their named counterpart organization "kept us informed of their progress," with only 41 percent of transportation Reviewers expressing agreement.
• 76 percent of both transportation and resource Reviewers agreed (or strongly agreed) that their named counterpart organization "was open and honest with us." Forty-nine percent of transportation Reviewers strongly agreed with this statement, in contrast to only 38 percent of resource Reviewers. This is the second highest level of agreement shown by both types of Reviewers to the 21 evaluation items, with only the first item ("My agency knew what was expected of it") receiving higher levels of agreement from both types.
• About half (53 percent) of transportation Reviewers and slightly more (58 percent) of resource Reviewers agreed (or strongly agreed) that their named counterpart organization "gave reasonable suggestions or alternatives" (item 15).
• Similarly, only 52 percent of transportation Reviewers and 55 percent of resource Reviewers agreed (or strongly agreed) that their named counterpart organization "was willing to compromise." Over one-quarter of transportation Reviewers Disagreed (or strongly Disagreed) with this statement about their resource counterparts.
• The biggest difference between transportation and resource Reviewers was observed on Exhibit item 18, where transportation Reviewers were 19 percentage points less likely than resource Reviewers to agree (or strongly agree) that their counterpart organizations "had adequate participation in key meetings." However, in this case, the response pattern of transportation Reviewers did not differ noticeably from its typical pattern (resembling the overall satisfaction responses). The observed difference arose because 84 percent of resource Reviewers agreed (or strongly agreed) with this statement. This difference, therefore, does not reflect an unusually negative perception by transportation Reviewers of their resource counterparts. Rather, it reflects the uniform views of resource Reviewers that the named transportation agencies participated adequately when key meetings occurred.

In general, a substantial majority of both types of reviewers appeared to be satisfied with interagency communications on their cited projects. On the summary item, "Overall, there was a sufficient level of communication between the two agencies on this project," there was strong similarity in the response distributions of both types of Reviewers, with approximately 70 percent in agreement, 20 percent giving a neutral response, and about 10 percent Disagreeing.

Regional Differences in Resource Reviewers' Evaluations

For the 10 items asked of transportation Reviewers, we found only two instances of significant differences in response patterns across the DOT Regions, items 12 and 18 in the Exhibit.

Transportation Reviewers gave their least positive evaluations in response to the question about whether the resource organization counterparts "kept us informed about their progress." Overall, 41 percent of transportation Reviewers agreed (or agreed strongly) with that statement. At the Regional level, transportation Reviewers varied considerably in their response distributions. In Region 10, 57 percent of Reviewers agreed with the statement, while only 19 percent Disagreed. By contrast, in Region 9 only 28 percent agreed with the statement, and 44 percent Disagreed. Agreement by transportation Reviewers in six other regions ranged from 35 to 40 percent, while in the two remaining regions, 52 percent agreed.

Wide regional variation was also found on evaluations of whether named resource organizations "had adequate participation at key meetings." Although the overall distribution closely mirrored that for the overall satisfaction measure, transportation Reviewers in Region 1 were far more positive than average, with 80 percent agreeing (or agreeing strongly) with the statement, in contrast to only 5 percent Disagreeing. At the other extreme, only 37 percent of transportation Reviewers in Region 2 agreed with the statement, while 47 percent Disagreed. (We note that Region 2 had only 19 transportation Reviewers responding to this item.) Reviewers in Region 9 were also significantly less positive than other regions, with only 45 percent agreeing with the statement, and 23 percent expressing Disagreement.

Regional differences for resource Reviewers were found on three communications-related items: numbers 10, 18, and 19 in the table below.

On the question asking whether named transportation agencies "responded in a timely way to our requests for information" (Exhibit item 10), across all DOT Regions, resource Reviewers exhibited a response pattern nearly identical to that expressed on the overall satisfaction question (two-thirds of Reviewers indicating a positive evaluation). Resource Reviewers in Regions 4 and 5 were even more positive, with about 80 percent agreeing (or strongly agreeing) with the statement. However, this view was not shared by resource Reviewers in Regions 2, 9, and 10, where, respectively, only 55 percent, 54 percent, and 48 percent agreed. In Region 2, 36 percent of the resource Reviewers Disagreed (or strongly Disagreed) with the statement, in contrast to only 11 percent Disagreeing across all regions.

Regional differences were also found for the item about whether named transportation counterparts "had adequate participation in key meetings" (Exhibit item 18). Overall, 84 percent of resource Reviewers agreed (or strongly agreed) with this statement. In Region 2 (with 9 Reviewers responding to this item), only 67 percent agreed with the statement. In two Regions, Reviewers were nearly unanimous in their responses, with 95 percent of Region 1 and 99 percent of Region 7 expressing agreement with the statement.

For Exhibit item 19, which addresses whether "there was a sufficient level of communication between agencies on the project," shows that 71 percent of all resource Reviewers agreed (or strongly agreed). However, in two Regions, resource Reviewers were significantly less likely to agree, with Region 9 at 59 percent, and Region 2 at only 46 percent agreement. On the other hand, in Region 7, 90 percent of resource Reviewers agreed with the statement, as did 81 percent of those in Region 3.

Quality and Quantity of Inter-agency Communications (items 9 through 19)

Reviewers' Evaluations of the Timeliness of Interactions and the Duration of the NEPA Project

The final four questions in this portion of the survey asked Reviewers to evaluate their named counterpart organizations on issues associated with the timeliness and duration of the project, including the extent to which the named counterparts adhered to project schedules, whether the counterpart gave their own agency sufficient time to complete its work, whether the project took a reasonable amount of time, and whether the project duration might have been shortened without compromising NEPA intent.

With respect to adherence to schedules (Exhibit item 20), there was strong consistency between transportation and resource Reviewers with 66 percent of both types agreeing that their named counterpart "adhered to schedules that were set throughout the process." As shown in the tables below, other categories also matched within 5 percentage points for both types.

Exhibit item 21, however, shows a greater divergence in Reviewers' evaluations. Fully 80 percent of transportation Reviewers agreed (or strongly agreed) that their named resource counterparts "gave our agency enough time to accomplish tasks." Only 6 percent Disagreed at any level, and only 14 percent gave a neutral response. Resource Reviewers were slightly less positive about the performance of their named transportation agency counterparts, with 68 percent agreeing (or strongly agreeing) and 12 percent Disagreeing. Despite these differences, both types of reviewers gave somewhat more positive evaluations to this item than they did to other items.

Transportation and resource Reviewers also showed remarkably similar response patterns to the statement, "the entire process took a reasonable amount of time." In this case, however, the levels of agreement are slightly lower than those expressed for most of the 23 items in Exhibit 6, with 58 percent of both types of Reviewers agreeing (or strongly agreeing) with the statement. Neither the percentage that Disagreed (or strongly Disagreed) nor the percent giving a neutral response were especially high for either group.

Finally, in response to Exhibit item 23, which asked whether "the process could have been shortened without compromising the intent of NEPA," the two types of Reviewers displayed not only large differences of opinion between the types of Reviewers, but also a strong division of opinion within each of the two groups. While 55 percent of transportation Reviewers agreed (or strongly agreed) with the statement, only 35 percent of resource Reviewers shared that view. Conversely, 32 percent of transportation Reviewers Disagreed (or strongly Disagreed) with this statement, while 46 percent of resource Reviewers also indicated Disagreement.

Regional Differences in Reviewers' Responses to Timeliness and Project Duration Items

Transportation Reviewers across the 10 DOT Regions did not show response distributions materially different from the overall patterns shown in the four tables below (Exhibit items 20 through 23).

However, resource Reviewers differed significantly across regions on the last two items, which addressed whether the entire project took a reasonable amount of time (Exhibit item 22), and whether the project could have been shortened without compromising NEPA intent (Exhibit item 23). On item 22, resource Reviewers in six of the Regions (3, 4, 5, 6, 7 and 8) indicated higher levels of agreement than the overall average for all regions, especially in Region 3, where 83 percent expressed agreement that the project took a reasonable amount of time, and Region 7, where 52 percent "agreed strongly." Resource Reviewers in Regions 1,2, 9, and 10 were notably less likely to agree that the projects were completed in reasonable time, especially in Region 2, where only 27 percent agreed with the statement at any level, and only 9 percent agreed strongly.

As noted above, significantly more resource Reviewers Disagreed (45 percent) than agreed (35 percent) with the statement that their cited NEPA project "could have been shortened without compromising the intent of NEPA" (Exhibit item 23). Resource Reviewers in three of the DOT Regions (namely, Regions 4, 6, and 9) gave responses that were similar to the overall average. Resource Reviewers in Regions 1 and 2 were considerably more likely to agree with the statement, with 61 percent and 55 percent agreeing, respectively. In fact, in Region 1, only 6 percent of resource Reviewers Disagreed with the statement.

However, in both Regions 7 and 8, only 19 percent of resource Reviewers agreed that the project could have been shorted without compromising NEPA intent. In Regions 3, 7, 8, and 10, over half of the reviewers Disagreed with this item, with over one-third of those in Regions 4 and 8 expressing strong Disagreement.

Overall, transportation Reviewers were 19 percentage points more likely than resource Reviewers to agree that the project could have been shortened without compromising NEPA intent. Not surprisingly, in 9 of the 10 DOT Regions, transportation Reviewers were more likely to agree, with the magnitude of differences ranging from only 9 percentage points in Region 2 to a high of 39 percentage points in Region 7.

However, in Region 1, 61 percent of the resource Reviewers agreed that their cited project could have been shortened, in contrast to only 30 percent of transportation Reviewers. We believe this atypical reversal of beliefs arose for two reasons. First, Region 1 had the smallest numbers of Reviewers of each type participating in the survey, with only 20 transportation Reviewers and 18 resource Reviewers responding to this questionnaire item. Second, because the participating Reviewers cited a substantial number of different projects as the focus of their answers to survey questions, the small numbers of Reviewers in Region 1 may be somewhat less likely to be reporting on the same projects than Reviewers in the other Regions, which could account for the atypical response pattern observed.

Timeliness of process interactions between organizations (items 20 through 23)

Reviewers of both types who agreed with the statement that "the process could have been shortened without compromising the intent of NEPA" (225 transportation Reviewers and 118 resource Reviewers) were then asked, "How could the process have been shortened?" in an open-ended format and coded according to primary content. The most common answers given by both transportation and resource Reviewers were the same:

• 26 percent of transportation Reviewers and 28 percent of resource Reviewers said, "more timely responses" from their designated counterpart organization;
• 16 percent of transportation Reviewers and 17 percent of resource Reviewers indicated, "better communication and/or direction" from their reported counterpart organization; and
• 7 percent of transportation Reviewers and 16 percent of resource Reviewers reported, "earlier involvement and coordination of project effort" by their mentioned counterpart organization.

Thus, with respect to opportunities for reducing the duration of NEPA projects, reviewers on both sides of the process have substantially the same analyses and prescriptions for the process improvements necessary reduce the length of the projects on which they've worked.

Additional Performance Evaluations of Counterpart Organizations

Transportation and resource Reviewers were then asked to rate the performance of their named counterpart organizations on eight additional dimensions related to the process and outcomes of their cited NEPA projects. For each dimension, reviewers were asked to assign a score on a 5-point scale, ranging from "poor" (score of 1) to "excellent" (score of 5). Results for each of the eight items are shown in Exhibit 7 below.

With respect to diagnosing areas of strong positive or negative performance as viewed by Reviewers on either side of the projects, these items provide some new information beyond that revealed by the overall satisfaction question and the 23 indicators discussed above. However, the response distributions for most of these items mirror the pattern of overall satisfaction ratings described earlier. On all of the attributes presented in Exhibit 7, there is little evidence of deep dissatisfaction by Reviewers of either type. The two lowest values (1 and 2) were typically chosen by a combined total of about 15 percent of Reviewers, and only for items 6 and 7 do the totals exceed 20 percent (for transportation Reviewers). The percentage of Reviewers giving "neutral" scores (a rating score of 3) typically ranges from 13 to 26 percent, and is slightly higher only for two of the eight items (again, items 6 and 7). Most commonly, about two-thirds of both types of Reviewers give rating scores of 4 or 5 to their named counterparts — very similar to the distribution of overall satisfaction scores.

For a few of the eight items asked of both types, we observe important statistical differences in the response patterns of transportation and resource Reviewers in ratings given to specific attributes in Exhibit 7. For example, on item 3 of the Exhibit, 76 percent of transportation Reviewers and 81 percent of resource Reviewers gave rating scores of 4 or 5 to the competence of the counterpart organization staff with whom they interacted. This strongly suggests that Reviewers do not view the capabilities and experience of counterpart staff as an impediment to successful NEPA processes. These are among the most positive ratings provided to any item in the survey.

Yet, we see that the responses of transportation and resource Reviewers diverged strongly to Exhibit item 7. Transportation Reviewers were much less favorable than resource Reviewers concerning whether their named counterpart organization devoted sufficient resources to their cited NEPA projects — only 48 percent of transportation Reviewers gave ratings of 4 or 5 on this item compared to 70 percent of resource Reviewers. This difference of 22 percentage points in positive evaluation is the largest observed in this group of eight rating scales and among the largest differences among all survey questions. Clearly, transportation Reviewers believe that the "level of resources," which may refer to level of effort by staff or to use of other organizational assets, is related to the perceived ability of resource-permitting organizations to provide timely responses during NEPA projects.

Both types of Reviewers also deviated from their typical response patterns on Exhibit item 7 concerning the range of reasonable alternatives the counterpart organization suggested for the cited project. On this item, only 42 percent of transportation Reviewers and 52 percent of resource Reviewers gave positive evaluations to their counterpart organizations, significantly less than the standard proportions. This is the lowest percentage of positive evaluations given by transportation Reviewers in this set of items — ranging from 20 to 34 percentage points lower than other items (except for Exhibit item 6 discussed in the preceding paragraph). It is also the lowest positive rating given by resource Reviewers in this group of 8 items, suggesting that significant proportions of both types of Reviewers believe that their counterpart organizations are not making adequate efforts to be flexible about the alternative solutions presented as solutions to project problems.

Exhibit item 9 shows the response distributions for resource Reviewers on the issue of the quality of effort by their transportation agency counterparts in "protecting the environment." Although the percentage distributions across the five rating values are not very different from the overall satisfaction measure, they are slightly less positive, with 58 percent indicating scores of 4 or 5, in contrast to an average of 65 percent on the prior 8 measures. Although the numbers are slightly lower, we do not consider a difference that should influence policy or to guide intervention planning.

We examined the response distributions to all of the survey items in Exhibit 8 for both transportation and resource Reviewers separately within each of the 10 DOT Regions. We did not find evidence of any statistically significant differences in response distributions across the DOT Regions.

Exhibit 8. Ratings by Reviewers of the Performance of Named Counterpart Organizations on Cited NEPA Projects

The following performance attribute was rated by resource Reviewers only:

Back to Table of Contents

Managers' and Reviewers' General Evaluations of Relationships with Counterpart Agencies

Transportation and resource officials (both Managers and Reviewers) were asked about their general perceptions of their counterpart agencies, based on their experiences and interactions over a longer time period (beyond the cited NEPA project cited by Reviewers discussed in the preceding sections). Both transportation and resource officials were presented with eight identical statements and asked to rate how strongly they agreed or Disagreed, using a five-point scale (where 1 = strongly Disagree and 5 = strongly agree). An additional item was tailored for each type of official; transportation officials were asked whether counterpart resource organizations were "committed to transportation improvements," and resource officials were asked whether their counterpart transportation agencies were "committed to protecting the environment." Exhibit 8 displays the percentage distributions for the two types of officials in both transportation and resource-permitting organizations for all nine items answered by responding officials.

Table 1. Understanding organizational missions

Transportation and resource Managers differed sharply in their beliefs about whether their named counterpart organization understood their own agency's mission. Nearly two-thirds (64 percent) of resource Managers agreed (or strongly agreed) with the statement, but less than half (48 percent) of the transportation Managers agreed at any level.

Nearly three-fifths of both types of reviewers expressed agreement with the statement — closer to the response pattern of resource Managers than transportation Managers.

With the exception of transportation Managers, relatively few respondents Disagreed that their counterpart resource organization understood their agency's mission. However, one-fourth of transportation Managers Disagreed with the statement.

Table 2. Caring about organizational missions

In responses similar to those in the previous table, transportation and resource Managers showed the greatest differences in levels of agreement with the statement that their named counterpart organization cared about their own agency's mission. Just over half of the resource Managers agreed (or strongly agreed) with the statement, in contrast to only one-third of transportation Managers. Roughly two in five Reviewers in both types of organizations agreed with the statement.

In relation to most of the other items in Exhibit 8, levels of Disagreement were relatively high on the issue of whether counterpart organizations "care about your agency's mission." The percentage that Disagreed (or Disagreed strongly) ranged from 21 percent for resource Managers, to 24 percent for resource Reviewers, 32 percent for transportation Reviewers, and a high of 38 percent of transportation Managers. These relatively negative general beliefs about counterpart organizations may indicate an opportunity for high-level management interventions to determine why these perceptions exist and to investigate possible approaches for improvement in this aspect of agency relationships.

Table 3. Commitment to doing quality work

Table 3 of Exhibit 8 shows that about four-fifths of both types of transportation officials agree (or strongly agree) that their named counterpart resource organization is committed to doing quality work. The levels of agreement are among the highest for any of the items in the Exhibit (together with Table 4 below). In this case, both transportation Managers and Reviewers were even more likely to agree (or strongly agree) than their resource counterparts and transportation Managers show the highest percentage agreement (they have the lowest percentages in agreement in all other tables).

Only about one in twenty respondents of any type expressed any level of Disagreement with this statement. One-sixth of transportation officials and about one-quarter of resource officials gave neutral responses. Respondents giving neutral scores may be suitable targets for management interventions to attempt to improve organizational perceptions and relationships.

Table 4. Competence of the counterpart organization's staff

Table 4 of the Exhibit also reveals exceptionally high levels of agreement by all types of respondents with the statement that their counterpart organization has competent staff, with two-thirds to three-quarters of respondents either agreeing or strongly agreeing. In this case, transportation Managers show slightly lower levels of agreement than other groups (66 percent), but the difference is considerably smaller than observed in all other tables (except Table 3).

As in Table 3, very few respondents expressed any level of Disagreement with the statement that their counterpart organization "has competent staff." As little as 21 percent and as much as 27 percent of each respondent group chose the neutral score.

Table 5. Level of trust between transportation agencies and resource-permitting organizations

Not surprisingly, the response distributions in Table 5 of Exhibit 8 are similar to those of Tables 1 and 2. Overall, about half of all responding officials expressed agreement (or strong agreement) that there is a sufficient level of trust between your two agencies. Resource Managers (58 percent) and resource Reviewers (52 percent) were slightly more likely to agree than transportation Managers (44 percent) and transportation Reviewers (46 percent).

Nearly three in ten transportation officials Disagreed (or strongly Disagreed) with the statement about the sufficiency of trust between agencies, a considerably higher level of Disagreement than that shown by resource officials.

Table 6. Commitment to making the environmental review process run efficiently

With the exception of transportation Managers, the response distributions for Table 6 of Exhibit 8 on whether the counterpart organization is committed to making the environmental review process run efficiently are virtually identical to those in Table 5 above. In this case, however, transportation Managers gave even less positive responses, with only 37 percent agreeing (or strongly agreeing) with the statement.

Conversely, 34 percent of transportation Managers Disagreed (or strongly Disagreed) with the statement — well over twice the percentage of resource Managers or Reviewers who expressed any level of Disagreement.

As for most other items in Exhibit 8: 26 to 33 percent of the four groups of officials gave neutral responses to the item on commitment to making the environmental review process run efficiently.

Table 7. Willingness to compromise

The pattern of responses in Table 7 to the statement about whether the counterpart organization is willing to compromise also resembles those for Tables 5 and 6. However, the percentages who express agreement (or strong agreement) are lower for all types of respondents, and are much lower for transportation Managers, of whom only 28 percent express any level of agreement with the statement — the lowest level of overall agreement from any group to any of the eight statements read to all respondents.

Transportation Managers were more likely than all other officials to express Disagreement (or strong Disagreement) with the statement about "willingness to compromise," with over one-third (35 percent) expressing Disagreement, again over twice the level of Disagreement from resource Managers.

Just over one-third of all categories of respondents (35 to 37 percent) gave neutral responses to the statement about whether their named counterpart organization was "willing to compromise."

Table 8. Levels of communication between agencies

The response distributions shown in Table 8, which reflect whether there is a sufficient level of communication between our two agencies, are quite similar for all four types of respondents, with the exception of transportation Managers, who are significantly less likely to express strong agreement with the statement. Nevertheless, well over half of the officials in each category expressed some level of agreement.

Transportation Managers and Reviewers were only slightly more likely than resource officials to Disagree that communication levels were sufficient.

About one-quarter of each group of respondents gave neutral responses concerning communication levels.

Tables 9. and 10. Commitment to transportation improvements and protecting the environment

The final two tables in Exhibit 8 show the agreement levels separately for transportation officials, who were asked to respond to the statement that their resource counterparts are committed to transportation improvements, and for resource officials, who were asked to respond to the statement that their transportation counterparts are committed to protecting the environment.

Tables 9 and 10 also display the asymmetry between transportation and resource officials seen in the other eight combined tables. In Table 9, transportation officials are less likely to express any level of agreement about the commitment of their resource counterparts to "transportation improvements." Only about one-fourth of transportation Managers agree at any level with the statement, and less than 10 percent agree strongly. At the opposite pole, 44 percent of transportation Managers Disagree (or Disagree strongly) with the statement, and just over 30 percent give neutral responses. Transportation Reviewers also express a lower level with this statement than with any of the eight other statements in Exhibit 8. Transportation Reviewers are equally divided between agreement (34 percent), Disagreement (33 percent), and neutral responses (33 percent).

Table 10 also shows relatively low levels of agreement and high levels of Disagreement on the part of resource officials about the commitment of their transportation counterparts to "protecting the environment" — although the distributions are quite close to those observed in Table 7, which focus on "willingness to compromise." Resource Managers were slightly more likely to express agreement and slightly less likely to express Disagreement with the statement than were resource Reviewers, a pattern seen in most of the other tables in this Exhibit.

In general, however, resource officials were considerably more likely to express agreement that their transportation agency counterparts were committed to environmental protection than transportation officials were to express agreement that their named counterpart resource organizations were committed to transportation improvements. In fact, the percentage of resource Managers expressing agreement was about twice the percentage shown by transportation Managers (26 percent vs. 49 percent). The extent of the difference between transportation and resource Managers is even greater at the opposite end of the scale, with 44 percent of transportation Managers Disagreeing, compared to only 19 percent of resource Managers.

With respect to the named counterpart agency's commitment to each respondent's goals, transportation and resource Reviewers also differed in the same pattern as their Managers, but the differences were considerably smaller (a ten percentage point difference in any level of agreement and a 9 percentage point difference in those expressing Disagreement).

Approximately equal percentages (31 to 33 percent) of all four types of officials gave neutral responses to the item on their named counterpart agency's commitment to their own goals.

Overall Patterns in Exhibit 8

A number of general patterns are apparent in the response distributions shown in the tables of Exhibit 8:

• With only one exception (Exhibit item 3 — whether the named counterpart organization is "committed to doing quality work"), transportation Managers were the least likely to express any level of agreement with the nine statements about their named counterpart resource organizations, and especially unlikely to express strong agreement.
• On six of the nine items asked of transportation officials, transportation Managers were the most likely to express Disagreement (especially strong Disagreement).
• Transportation Reviewers generally gave more positive and less negative responses than transportation Managers. For several of the items (1, 2, 4, 6, and 7), the response distributions of transportation Reviewers are closer to those of resource Reviewers than they are to those of transportation Managers.
• With the exception of Exhibit item 3, both types of resource officials typically gave more positive responses than did both types of transportation officials.
• In most of the tables, resource Managers were more likely than other three types of respondents to agree (or agree strongly) with the statements about their counterpart transportation agencies, although the observed differences are sometimes small.
• In seven out of eight tables in Exhibit 8 (with item 3 as the lone exception), the largest differences in response patterns are evident between transportation Managers and resource Managers. Transportation and resource Reviewers show more similar distributions in these seven tables.

Exhibit 9. Perceptions of Managers and Reviewers of the General Performance of Named Counterpart Organizations on all NEPA Projects on Which They Have Interacted

Back to Table of Contents

On the following page, Exhibit 10 displays the mean (average) scores on the five-point agree/Disagree scale used for the items analyzed in Exhibit 8 above. Of the 36 cells containing mean values in Exhibit 10, 28 (78 percent) fall in the range from 3.0 to 3.9; three (8 percent) are 2.9 or lower, and five (14 percent) are 4.0 or higher.

The patterns in mean scores for the four respondent groups generally mirror the discussion of percentage distributions. With one exception (item 3), transportation Managers have the lowest average scores, followed by transportation Reviewers with mean scores that are typically 0.2 to 0.3 higher than those for transportation Managers.

Except for items 3 and 4, average scale scores for resource Managers and Reviewers are typically very close, never differing by more than 0.1 units, and means for resource Managers are typically 0.4 or more units higher than those for transportation Managers.

Exhibit 10. Comparison of Mean (average) Agree/Disagree Scale Scores for Counterpart Organization Ratings

	Item	Transportation Managers	Resource Managers	Transportation Reviewers	Resource Reviewers
1	[Named counterpart organization] understands your agency's mission	3.3	3.7	3.6	3.7
2	[Named counterpart organization] cares about your agency's mission	2.9	3.4	3.1	3.3
3	[Named counterpart organization] is committed to doing quality work	4.0	4.0	4.1	3.9
4	[Named counterpart organization] has competent staff	3.8	3.9	4.0	4.0
5	There is a sufficient level of trust between your two agencies	3.2	3.6	3.3	3.5
6	[Named counterpart organization] is committed to making the environmental review process run efficiently	3.0	3.6	3.3	3.5
7	[Named counterpart organization] is willing to compromise	2.9	3.4	3.2	3.4
7	[Named counterpart organization] is willing to compromise	2.9	3.4	3.2	3.4
8	There is a sufficient level of communication between your two agencies	3.5	3.7	3.6	3.8
9	[Named counterpart organization] is committed to transportation improvements	2.7	N/A	3.0	N/A
10	[Named counterpart organization] is committed to protecting the environment	N/A	3.4	N/A	3.3

Back to Table of Contents

Status and Change in Inter-Agency Relationships

Following the survey questions on individual dimensions of inter-agency relationships, transportation and resource officials were asked to provide a general characterization of their agency's relationship with their named counterpart organization, using a five-point scale with end-points labeled "1" for "Poor" and "5" for "excellent." Exhibit 10 displays the response distributions for the four types of respondents.

Exhibit 11. Overall Evaluations by Transportation and Resource Officials of the Current Relationship with Their Named Counterpart Organizations

Back to Table of Contents

The distributions in Exhibit 11 provide a predictable summary of the responses presented in Exhibit 8. Transportation Managers were least likely to give responses of "4" or "5" — only 47 percent, in contrast to 61 to 63 percent of the other three categories of respondents. Only 7 to 8 percent of resource officials gave scale scores of "1" or "2;" while the comparable percentages for transportation officials were exactly twice as high. As for most items in Exhibit 8, 30 to 39 percent of respondents in all categories gave neutral responses to this item.

Respondents were also asked whether the relationship with their named counterpart organization had "improved, stayed the same, or gotten worse" over the previous three years. Exhibit 12 shows the response patterns for the four respondent groups.

Exhibit 12. Change in Relationships with Their Named Counterpart Organizations Perceived by Transportation and Resource Officials during the Last Three Years

Back to Table of Contents

The asymmetric pattern between transportation and resource officials is still visible in Exhibit 12, but to a somewhat lesser degree. Both transportation Managers and Reviewers were about twice as likely as their resource counterparts to report that the relationship had gotten worse over the last three years (14 to 16 percent for transportation officials in contrast to 7 percent for resource officials). About one-third of all four respondent groups (33 to 38 percent) said that the relationship had stayed the same.

Roughly half of the respondents (nearly three-fifths of transportation Reviewers) reported that inter-agency relationships had improved — a very encouraging trend for the environmental streamlining process. As seen in most other tables in this report, resource officials were slightly more likely than transportation officials to report improved relationships, but the differences were relatively small — 55 to 59 percent for resource Managers and Reviewers respectively, in contrast to 51 and 49 percent for transportation Managers and Reviewers.

Expectably, there is a strong positive correlation between the response distributions for the overall characterization of the relationship and the perceived change in the relationship during the last three years for all four categories of respondents:

• Those who gave lower scores of "1" or "2" to the quality of the current relationship were much more likely than others to report that the relationship had gotten worse over the last few years;
• Those who gave higher ratings of "4" or "5" to the current relationship were much more likely than others to indicate that the relationship had improved during the prior three years; and
• Those who gave the neutral response of "3" to the overall relationship rating were much more likely to report that the relationship had stayed the same, and also somewhat more likely to report that it had improved.

We believe that an inference that the current overall ratings of counterpart agencies by Managers and Reviewers in both types of organizations are influenced by trends in interactions over recent years is supported by these survey data. If supported by further qualitative research, this inference would provide a strong basis for a management strategy of building stronger long-run relationships between transportation and environmental agencies by pursuing a sequence of small but continuous intermediate steps that would build greater trust and confidence between pairs of agencies that currently interact in an excessively adversarial style.

Further Implications for Management

The results in Exhibits 8 and 9 focus on more general aspects of the relationships among transportation agencies and resource-permitting organizations than those in preceding Exhibits, which focused on specific NEPA projects cited by agency Reviewers who performed day-to-day work on those projects. In their responses to items on specific projects, in most cases transportation and resource Reviewers responded in patterns similar to the overall satisfaction ratings they gave to working with their named counterpart agencies on their cited NEPA projects, where, for both types of Reviewers:

• almost two thirds indicated they were satisfied or very satisfied,
• less than one-sixth indicated they were dissatisfied or very dissatisfied, and
• one-fifth gave neutral responses.

With the exceptions noted above, Managers and Reviewers in both types of organizations were less likely to give positive scores to their counterpart agencies on more general aspects of their relationships, such as understanding and caring about their agencies' missions, levels of communication and trust between agencies, and especially on such matters as being willing to compromise and committed to making the environmental review process run efficiently. (The exceptions were the items focusing on commitment to doing quality work and having competent staff.) Transportation Managers were consistently more negative in their perceptions of resource agencies than the Reviewers who work on NEPA projects on a day-to-day basis. These managers show exceptionally high levels of Disagreement with the notion that their counterpart resource agencies are committed to transportation improvements.

On the other hand, resource Managers give slightly more positive ratings to counterpart transportation agencies than day-to-day Reviewers on NEPA projects, but these differences are relatively small. Although resource officials tend to exhibit more positive perceptions about their transportation agency counterparts than vice versa, resource Managers and Reviewers also showed their most negative perceptions on the survey item asking whether their named counterpart transportation agencies were committed to protecting the environment.

The reasons for these differences in perceptions among transportation and resource officials cannot be readily determined from the survey data. The negative beliefs shown by both sides about their counterparts' lack of commitment to their own primary goals may have been formed as a result of repeated experiences on a wide variety of NEPA projects over several years. Alternatively, these views may have been strongly affected by a single, recent, and especially difficult or problematic project, leading to generalizations about motives and intentions that may not be based entirely on experience.

Moreover, the perception of a transportation official that an environmental resource-permitting agency is not committed to transportation improvement may not be intended by that official as a critical or negative evaluation, but simply as a perception that, despite all the recent provisions designed to streamline the process, the current NEPA system remains an adversarial, conflictual process in which both sides are encouraged, if not required, to pursue their primary goals aggressively, and to achieve their desired outcomes using all available means — at the expense of their opponents, if necessary.

Because this survey is the first to measure the perceptions of the participating officials, we cannot know how the response distributions reported here compare to what they might have been prior to the initiation of environmental streamlining practices. However, from these baseline measures, we believe there are several major findings that should assist management in decision-making and potential policy or procedural interventions:

1. At the present time, there is a considerable degree of satisfaction among officials on both sides of the process with the way the system currently operates, both on specific projects and in overall agency relationships;
2. Levels of satisfaction with current projects and organizational relationships vary considerably.
   • Perceptions vary across process and performance domains, with
     • more positive perceptions of staff competence and organizational commitment to quality of process;
     • more neutral or moderate evaluations of process matters, such as levels of communication, investment of resources in meetings and interactions, involvement of specific parties at appropriate stages of the process,
     • generally less positive perceptions about such matters as timeliness of response and quality/quantity of information provided, willingness to compromise and to consider mitigation alternatives, and
     • the most negative perceptions and the least agreement between the two sides on factors related to strategic and tactical considerations, such as regard for the other side's organizational mission, recognition of the legitimacy of and commitment to the other side's primary goals, and such basic matters as how much time a project should take in order to fulfill NEPA intent;
   • Perceptions vary by the transportation official's role in the process, with higher level transportation Managers showing significantly more negative perceptions and evaluations than day-to-day transportation Reviewers;
3. The tables show there is considerable room for improvement in respondents' perceptions and ratings for all but a few of the performance and process dimensions measured in the baseline survey and reported above. By "improvement," we mean changes in the project interactions and inter-agency experiences and relationships that will lead future survey respondents on both sides to give more positive rating scores to their counterpart organizations than those observed in the baseline survey. In the context of the baseline measurement design, this will require management action that leads to improvements that will be reflected in respondents changing their perceptions from lower values of 1 through 3 on the five-point scales used in this survey to values of 4 and 5.

In practical terms, ratings of such dimensions as staff competence and commitment to doing quality work are already so high that further improvements on these attributes will be difficult to achieve. At the other extreme, for most of the performance measures related to specific NEPA projects, there are relatively few dimensions in which we observed high percentages of dissatisfied Reviewers. Nevertheless, straightforward management interventions may be developed to address and eliminate specific project problems that could improve Reviewers perceptions of their project interactions with counterpart organizations and, as a result, also translate into more favorable perceptions of the overall relationships between agencies.

In addition, we believe that structured interviews and focus groups with transportation Managers could provide fuller understanding of the reasons for their less positive perceptions of overall relationships with environmental organizations and to generate specific ideas for actions that would address their concerns. Additional interviews and focus groups with resource Managers would allow preliminary discussion of new process management strategies in an effort to prevent downturns in their relatively positive perceptions of inter-agency relationships.

In most of the tables of Exhibit 8, one-fifth to one-third of respondents typically gave neutral responses, thus neither agreeing with nor Disagreeing with the statements about their counterpart organizations. This is a significant proportion of the transportation and resource officials interviewed in the survey, and it may seem a reasonable group to target for management interventions to improve inter-organizational perceptions.

However, because the neutral response category conveys little explicit content, it is difficult to interpret its meaning. For example, in the case of Table 4 of Exhibit 8 (whether the counterpart organization has competent staff), possible interpretations of a neutral response are

• the respondents do not have adequate information about counterpart staff competence to either agree or Disagree,
• respondents feel the counterpart organization has a varied mix of staff, some highly competent but others less so, or
• respondents believe that the counterpart organizations' staff uniformly perform at an adequate level of competence that is perceived as neither high nor low.

Additional data are needed to plan a strategy for improving perceptions of those who gave neutral responses. As noted above, the correlation between positive perceptions and improving trends in inter-agency relationships may be especially useful as a guide to planning interventions that lead to a succession of small but successful interactions that will eventually change perceptions and the underlying processes and results on NEPA projects.

Recommendations for Management

We recommend two types of action steps for consideration by FHWA based on the results of the 2003 survey presented above:

• policy and process interventions that will improve the review process on transportation projects, leading to more positive ratings of counterpart organizations by both transportation and resource officials, and
• modifications and additions to the evaluation design used in the baseline survey that will provide increased guidance for specific policy and process interventions in future years.

Improving Review Processes, Outcomes, and Resulting Evaluation Ratings

The 2003 survey results reveal that while the majority of participants and stakeholders in the Environmental Streamlining process are satisfied with most aspects of the overall relationships and project-focused interactions with their counterpart agencies, there remain many opportunities for improvement. Several of the survey questions that evinced the most negative ratings by Managers and Reviewers focused on respondents' views concerning the understanding and acceptance of their agencies' mission and role by the counterpart organizations.

Among the clearest findings are the beliefs among many transportation officials that resource agencies do not evince a commitment to transportation improvements, and among resource officials that transportation agencies have little commitment to protecting the environment. The prevalence of these beliefs may indicate that environmental reviews of transportation projects are intended to be adversarial in nature, and that the absence of shared commitments with the "other side" is both expected and considered appropriate.

The interventions described below may be effective in addressing these sources of disharmony between transportation and resource agencies:

• Conduct additional focus group sessions with transportation and resource agency Managers. The largest differences in evaluation ratings of inter-agency relationships observed in the 2003 survey were the contrasts between transportation and resource Managers. These differences were consistently larger than those found between transportation and resource Reviewers. Left unaddressed, the divergent attitudes of senior Managers may, over time, affect the experiences and perceptions of Reviewers, possibly obstructing improvements in relationships among technical staff who perform day to day work on transportation projects. Focus groups and/or in-depth structured interviews with transportation and resource Managers, individually and jointly, would provide a fuller understanding of the reasons for their more divergent perceptions of the overall relationships. Such sessions would also be a rich source of ideas for specific interventions to address their concerns.
• Initiate and/or enhance programs to promote inter-agency understanding and collaboration. Training, workshops, conferences and other joint activities are already incorporated into FHWA's plans for promoting the goals of Environmental Streamlining. The 2003 survey findings may reflect the positive effect of these activities to date. Nevertheless, the survey data show repeatedly that approximately one-third of Reviewers and about two-fifths of Managers continue to hold differing expectations concerning how transportation review projects should run and how their counterpart agencies should perform. Views of Managers and Reviewers in the two types of organizations are even more divergent concerning how much time the projects should take to be completed, and whether there are significant opportunities to reduce their duration without compromising NEPA intent. Using the results of the focus groups suggested in the previous paragraph, we recommend that FHWA continue and expand programs and activities designed to assist participants and stakeholders in developing common frames of reference, expectations (especially about project duration), and criteria for evaluating the performance of their counterparts. Causing change in deeply ingrained views among agencies that have been adversaries for decades will require sustained high levels of promotional activity that clearly communicate the guidance of senior leadership in both transportation and environmental organizations.
• Design training and promotional efforts around the evaluation measurement system. Inherent in FWHA's performance evaluation approach is the assumption that measurement will enable desired changes in process and outcomes to be achieved. It is essential that participants and stakeholders in the Environmental Streamlining process have full awareness of the program goals and the measurement system that will allow FHWA and other agencies to assess progress toward those goals. We recommend that part of the promotional effort include training on the evaluation measures, the survey findings (baseline results and future change measures), and the policy and process interventions instituted by agency leaders based on the evaluation results.
• Find and exploit "best practice" methods and approaches used by the most effective practitioners. On most of the evaluation measures discussed above, more than three-fifths of the officials in both types of agencies expressed approval of or satisfaction with their relationships and interactions with counterpart organizations. On many items, about one quarter of the transportation and resource officials expressed the highest possible level of satisfaction with and regard for their counterpart agencies. Gallup believes strongly that one of the best opportunities to improve performance on key organizational goals is a systematic program that
  • identifies the organizational workgroups and other entities that are performing at the highest levels,
  • studies and understands their approach, methods, techniques, and operating style,
  • implements methods to communicate these effective strategies and methods throughout the organization, and
  • engages the most successful practitioners as mentors or collaborators with other parts of the organization whose current performance is not meeting established standards.
  We recommend that FHWA consider identifying and conducting additional focus groups or one-on-one structured interviews with officials in both types of organizations who have achieved the strongest inter-agency relationships and project interactions. We are confident that findings from this qualitative research will identify many "best practices" that may communicated to and adopted by agencies in other locations.

Improving the Evaluation and Measurement Design

Although the design, conduct, and analysis of the initial Environmental Streamlining evaluation study met all objectives under the contract, our analyses of the baseline data suggest that there may be significant benefits to FWHA to altering the design and measurement approach.

• Develop a comprehensive list of major ongoing transportation projects undergoing environmental review in each DOT Region. In the absence of a comprehensive mapping of inter-agency relationships and major transportation projects, for the 2003 survey Managers and Reviewers were asked to identify a recently completed project on which they had the most interaction with a counterpart organization, and also to name that organization. Although this approach ensured that survey respondents would be providing evaluations of projects and agencies with high salience and high levels of interaction, it has the disadvantage that, in any Region, transportation and resource officials were likely to be providing evaluations about different projects. The lack of common project focus limits our ability to make inferences about the similarity or divergence of the views of transportation and resource officials. The observation, for example, that transportation and resource Managers often showed highly divergent satisfaction levels of their counterpart organizations may be explained by the fact that they were focusing on different project and different agencies and not actually evaluating each other, despite their co-location in the same Region. To improve understanding of the outcome of efforts undertaken in each Region to encourage agencies to collaborate and work toward common expectations and standards, the study design should be based on a comprehensive list of projects so that practitioners in each Region could be interviewed about the same project. Focusing on specific projects would also permit the incorporation of other information about the project (e.g., its actual duration in years and months, its potential economic value, and many qualitative features). Designing future studies around a common project focus (within Regions) would permit more detailed interpretations of the patterns of responses given by officials in both types of agencies.
• Establish a continuous measurement program. The 2003 survey has provided FHWA with a baseline measure for the attributes overall and for the ten regions. To measure progress against goals, the measurement system must be updated at reasonable intervals. Continuous measurement will permit tracking of trends in practitioner attitudes and beliefs over time, permit analysis of differences in evaluations that may be related to the particular mix of project types in the regions in a specific time period, and permits FHWA to assess change in inter-agency relationships and interactions in response to policy and/or process interventions designed to produce specific changes. To maximize the benefits to FHWA, Gallup recommends repeated measurement on an annual basis. These data will be used to provide empirically derived feedback for the training and promotional efforts.
• Review and Revise Questionnaire Items as Appropriate. Although we heed the well-established warning to "avoid changing the measure when trying to measure change," we also believe that the practical value of revising the initial measures may have large payoffs downstream. We recommend consideration of the following three measurement issues:
  • Assessing the occurrence of significant project problems. Just over one-third of Reviewers in both types of agencies indicated that they had experienced significant problems on their cited projects. We observed considerable differences among transportation and resource Reviewers in the phases of the projects where problems arose and in the nature of the problems as described by reviewers. Although the mention of project problems correlated highly with the expression of dissatisfaction of the counterpart agencies' performance, too little information was collected in the survey about the nature of the problems and the project on which it/they occurred to use this information for diagnostic purposes or to develop improvement strategies. We recommend that consideration be given to expanding this section of the survey questionnaire to collect more detailed information about the number and nature of project problems and Reviewers' perceptions of how such problems might have been avoided.
  • Common project focus for reviewer evaluations. As noted above, in the 2003 survey, respondents were asked to select a project on which they had interacted most with counterpart agencies, about which their perceptions were measured. However, the specific project was not identified by name. To increase the power of the survey data for intensive analyses, we recommend that a small number of key NEPA projects be pre-identified for each DOT Region.
  • Respondent participation in programmatic activities designed to improve the Environmental Streamlining process. Future surveys will have the opportunity to assess whether additional inter-agency efforts to promote collaboration and coordination on NEPA projects have effects on the perceptions of participants and stakeholders on the quality and efficiency of the projects. We recommend adding a small number of items to the survey questionnaire to measure each respondent's direct involvement and/or participation in these promotional activities.