Background
In 2003, The Gallup Organization, on behalf of the Federal Highway Agencies (FHWA), conducted a baseline measurement to better understand the workings of resource and transportation agencies in streamlining their environmental review processes. Environmental streamlining objectives, as first described in the Transportation Equity Act for the 21st Century (TEA-21) in June, 1998 were to:
1. Establish an integrated review and permit process that identifies key decision points and potential conflicts as early as possible
2. Integrate the National Environment Protection Act (NEPA) process as early as possible
3. Encourage full and early participation by all relevant agencies that must review a highway construction project or issue a permit, license, approval, or opinion relating to the project
4. Establish coordinated time schedules for agencies to act on a project
Today, three years after the baseline survey and two transportation bills after from ISTEA, FHWA continues to work on improving the environmental streamlining process. In 2006, Gallup was hired to conduct a second wave of the streamlining measurement survey to assess any changes that occurred during this time.
Overview
The Gallup Organization has undertaken a second wave of the environmental streamlining survey to provide comparative data for regions to gauge their own performance. In order to provide research comparisons for the two time periods, the Gallup/FHWA team changed the survey instrument very little (see the Questionnaire section for a discussion of changes). Therefore, comparison data is provided in all charts and graphs shown in the sections that follow.
In addition, Gallup was contracted to conduct a special sub-sample for five individual states providing them each with their own scorecard and report. The states who chose to participate in the survey included California, Pennsylvania, Illinois, Florida, and Texas. To participate, each state had to provide Gallup with a list of qualified NEPA reviewers from both resource and transportation groups in their states.
Questionnaire
The questionnaires used for these surveys were almost identical to the survey instruments used in the 2003 study so that comparison data could be presented throughout the reports. As in the past, two survey instruments were written with as few differences as possible between the two. Only a few differences were undertaken from the 2003 survey as indicated below:
Questionnaire Changes From 2003
SCREENER CHANGED: One change was made to the previous 16 screener questions used in the 2003 survey so that only 15 screener questions were asked. In 2003, Gallup asked if respondents had been involved in a NEPA project within the last 90 days. If they said no, the question was expanded to see if the respondent had been involved in a NEPA project in the past six months. In the 2006 survey, respondents were only asked if they had been involved in a NEPA project in the past six months.
OPEN-ENDED QUESTION ADDED: Q6B1 was added the questionnaire in 2006. The question follows Q6B which asks, [AGENCY] provided my agency with the materials, information, or documentation that we needed. on a scale of "1" to "5" with "5" being strongly agree and "1" being strongly disagree. If the respondent answered a "1" or "2" to Q6B, the respondent was then asked the additional question of Q6B1, What specifically were you missing? The question was open-ended and two responses were allowed. Gallup coded the responses received.
QUESTION WORDING CHANGED: Q8D was difficult for respondents in the 2003 survey to understand. As a result, the Gallup/FHWA team changed the question so that it was easier to understand. The original question was, The process could have been shortened without compromising the intent of NEPA. In 2006, the new question is, The process was the shortest it could have been without compromising NEPA. As a result, direct comparisons between these two questions cannot be made. However, the question continues to be part of an index used in both the 2003 and the 2006 analysis.
QUESTIONS DELETED: Three questions were deleted in the 2006 survey as further analysis found the questions to be repetitive and highly correlated with other questions on the survey. As such, the decision was made to delete them. All three questions came from the "performance" section of the survey:
Q9C. The competence of their agency staff who you interacted with (similar to 11D)
Q9D. Their ability to stay organized throughout the project (similar to 8A)
Q9E. Their understanding of your agency&146;s requirements (similar to 11A)
OPEN-ENDED QUESTION ADDED: A final open-ended question was added to the 2006 survey which followed up Q13 which asked, In general, how would you rate the overall relationship between your agency and [AGENCY]? The open-ended question, Q14, was asked of all respondents, If there was one thing that [AGENCY] could do to improve the relationship, what would it be?
No other changes were noted between the 2003 and 2006 questionnaires for either the resource or the transportation questionnaire.
Questionnaire Layout
The six main sections from 2003 remained the same and included problem occurrence, relationships, communication, timeliness, performance, and general. Four open-ended questions were asked and the complete set of responses to those open-ended questions are in Appendix 2 of this report. A verbatim analysis of the question is also contained within this report.
Problem Occurrence
The problem occurrence section first asked respondents if they had experienced a problem during the course of this project. For those who said "yes" they had experienced a problem, a follow-up set of eight areas of the NEPA process followed. As in 2003, respondents were asked about each of the eight areas and so could let the interviewer know that they had experienced a problem in more than one area mentioned. The eight areas queried included:
1. Early project planning
2. Defining purpose and need
3. Information or data collection
4. Development and analysis of alternatives
5. Analysis of impacts
6. Selection of preferred alternatives
7. Commitment to mitigation measures
8. Finalizing documents or response to comments
A follow-up question asked those who indicated they had experience a problem to briefly summarize what that problem was. An analysis of those verbatim comments is included in this report.
Relationship Questions
Seven questions asked respondents about relationship issues they had with their sister agencies. The questions asked in the relationship question were a subset of the Gallup Q12 times that focus on relationships in the workplace. The relationship questions included:
1. My agency knew what was expected of it in this process
2. [AGENCY] provided my agency with the materials, information, or documentation that we needed
3. [AGENCY] wanted to play an active role in the process
4. My agency&146;s opinions seemed to count in the permitting process
5. [AGENCY] felt the mission of this project was important [ASKED ONLY OF TRANSPORTATION REVIEWERS]
6. [AGENCY] felt the mission of this project was important
7. [AGENCY] helped to move this project forward
8. [AGENCY] made efforts to improve the process during this project
In addition open-ended questions were asked after Q2 above. For those respondents who answered a "1" or a "2" on a "5" point scale, a follow-up question asked, What specifically were you missing. Respondents were allowed to mention up to two items. The results of that follow-up question are included in this report. At the end of the survey, two additional relationship questions were added that asked both reviewers and managers to consider their overall relationship with their sister agencies:
9. In general, how would you rate the overall relationship between your agency and [AGENCY]?
This question was followed-up by an open-ended question that asked all respondents:
10. If there was one thing that [AGENCY] could do to improve the relationship, what would it be?
Communication Questions
Eleven questions that compromise the communication questions changed slightly for transportation vs. resource reviewers. They included:
1. [AGENCY] involved us early on in the process [ASKED ONLY OF RESOURCE REVIEWERS]
2. [AGENCY] responded in a timely way to our requests
3. [AGENCY] invited our participation in key meeting
4. [AGENCY] kept us informed of their progress
5. [AGENCY] was open and honest with us
6. [AGENCY] was open to our suggestions or alternatives
7. [AGENCY] gave reasonable suggestions or alternatives
8. [AGENCY] gave clear explanations if they did not agree with our
9. [AGENCY] was willing to compromise
10. [AGENCY] had adequate participation at key meetings
11. Overall, there was a sufficient level of communication between the two agencies on this project
Timeliness Questions
The four questions that comprise the timeliness questions were identical for both resource and transportation agencies.
1. [AGENCY] adhered to schedules that were set throughout the process
2. [AGENCY] gave your agency enough time to accomplish tasks
3. The entire process took a reasonable amount of time
4. The process was the shortest it could have been without compromising NEPA
After the final question was asked, respondents who answered a "3", "2", or "1" were asked a follow-up open-ended question, How could the process have been shortened without compromising NEPA? All responses were coded and results are contained within this report.
Performance Questions
There were six questions in the performance set of questions with one question specifically for resource reviewers.
1. The quality of information [AGENCY] provided to your agency
2. The completeness of information they provided
3. The level of resources they devoted to this project
4. The range of reasonable alternatives they suggested for this project
5. Their willingness to consider a range of mitigation measures
6. How good of a job they did at protecting the environment [RESOURCE REVIEWERS ONLY]
General Questions
The last set of questions was considered to be general. Because they were not asked with a specific project in mind, both managers and reviewers were given these questions. The analysis within this report is broken down separately by manager and reviewer.
1. [AGENCY] understands your agency&146;s mission
2. [AGENCY] cares about your agency&146;s mission
3. [AGENCY] is committed to doing quality work
4. [AGENCY] has competent staff
5. There is a sufficient level of trust between your two agencies
6. [AGENCY] is committed to making the environmental review process a timely one while ensuring environmentally sound projects
7. [AGENCY] is willing to compromise
8. There is a sufficient level of communication between our two agencies
9. [AGENCY] is committed to protecting the environment [RESOURCE REVIEWERS/MANAGERS ONLY]
10. [AGENCY] is committed to transportation improvements [TRANSPORTATION REVIEWERS/MANAGERS ONLY]
A final question asked respondents their impression, over time, of their relationship with their sister agency.
11. Over the past three years, has your agency&146;s overall relationship with [AGENCY] improved, stayed the same, or gotten worse?
By phone, the questionnaire averaged 15 minutes total. The average amount of time respondents took for each survey was 11.5 minutes with up to an additional 5 minutes to complete the snowball effect. By Web, the survey took a comparable amount of time although no time was kept on those surveys as respondents could come and go out of the survey as they pleased.
Sampling Plan
Gathering Names
Because no listed sample of eligible NEPA reviewers and managers exists, the population of eligible respondents was constructed using several techniques. First, Gallup gathered the list that was used in the 2003 survey (gathered through Web, referral, and snowballing techniques). Every name on that list was called to assess if the previous respondent was still eligible for this survey. Once that list was generated, Gallup began calling all 50 states to get names of eligible transportation and resource reviewers and managers. The calling took place over a five-month period. In addition, the five individual states who participated in the survey process separately gave Gallup a list of eligible respondents from their respective states.
The list of Transportation and Resource Reviewers and managers for both 2003 and 2006 were generated by Gallup based on information that existed at the time the surveys were undertaken. It is understood that although they may not represent the entire population of Transportation and Resource Reviewers, they both represent the best list of reviewers that could be constructed. In the absence of any other lists to represent these populations, these lists were treated as the target population for this study. Results based on the data presented in this report, therefore, related to these lists only and may not be generalizable to any other populations of Transportation and Resource Reviewers.
Respondent Eligibility
To determine if a respondent was eligible for the survey, the respondent had to screen through 15 questions including:
· Have they worked with NEPA at this agency for at least two years?
· Have they been directly involved with obtaining or reviewing NEPA documents in the last six months?
· Can they choose a project that they&146;ve worked on with the most interaction with other resource/transportation agencies?
If the respondent could answer positively to all these questions, they qualified for the survey and were taken through it.
A general list of agencies, both resource and transportation, are listed below.
Transportation agencies include the Federal Aviation Administration, Federal Highway Administration, Federal Transit Administration, State Department's of Transportation, and City/County Transportation Authorities.
Resource agencies include Historic Preservation Agencies, Indian Tribes, State Departments of Natural Resources, State Departments of Environmental Quality/Protection, Bureau of Land Management, Coastal Commissions, U.S. Army Corps of Engineers, U.S. Environmental Protection Agency, U.S. Fish & Wildlife Service, Local Forest Preserve Districts, State Departments of Cultural Affairs, State Deparments of Parks and Planning Commissions, State Board of Water and Soil Resources, State Pollution Control Agency, National Marine Fisheries Service, National Park Service, Natural Resources Conservation Service, Tennessee Valley Authority, Tennessee Wildlife Resources Agency, U.S. Coast Guard, and the U.S. Department of Agriculture.
Snowball Sampling Technique
The snowball technique is a type of nonprobability sample. A nonprobability sample involves personal judgment somewhere in the selection process. The fact that the elements are not selected through probability methods precludes an assessment of sampling error or an inability to place bounds on the precision of the estimates. In this case, we do not know the population of respondents. Therefore, precision estimates will not apply although the sample but will include the most complete set of respondents known to be directly working with NEPA in both the federal and state governments.
The snowball sample is a judgment sample that is sometimes used to sample special populations. This sample relies on the researcher&146;s ability to locate an initial set of respondents with the desired characteristics. These individuals are then used as informants to identify others with the desired characteristics &133; those initially asked to participate would also be asked for names of others whose cooperation would be solicited. Thus, the sample "snowballs" by getting larger as participants identify still other possible respondents.
Impact of Weighting vs. Not Weighting the Data
The analysis of the 2006 FWHA survey is based on unweighted data. In order to determine if weighting of survey data would have had any impact on the estimates (the means of the variables of interest), the 2006 data were weighted and the estimates of both weighted and unweighted data were compared. This note describes the weighting procedure that was used for this purpose and summarizes the findings.
For each of the four groups (Transportation Reviewers, Resource Reviewers, Transportation Managers, and Resource Managers), the sampling was done independently within each state. These sampling strata (states) were also chosen as the weighting adjustment cells for each of these groups. The weighting procedure within each state was similar. The weight assigned to each completed survey within a particular state was the ratio of the estimated number of eligible entities to the corresponding number of completed surveys from that state. So, the group of completes were projected to represent the estimated number of eligible entities from that state. The latter quantity (estimated number of eligible entities) was derived as follows. All sampled entities from a state were divided into three groups:
E &151; known eligibles; i.e. the completed surveys
I &151; known ineligibles; i.e. the sampled units that were identified as ineligibles during data collection
U &151; unknown eligibles; i.e. sampled entities that didn&146;t fall into one of the other two categories
Note that only completed surveys could be identified as Eligibles. There were no eligible entities for which surveys were not completed. The numbers for CATI and Web surveys were combined to derive the three numbers (E, I, and U) for any state. The eligibility rate (ER) &151; the proportion of eligibles within the group of entities for which eligibility could be established &151; was calculated by dividing the number of eligibles (completed surveys) by the sum eligibles and ineligibles. So, ER = E/(E + I). The estimated number of eligibles (EE) was then derived as EE = E + U * ER. In other words, the estimated number of eligibles was the sum of the number of known eligibles and the estimated number of eligibles from the Unknown eligible group. Finally, the weights assigned to any completed survey within a particular state were calculated as the ratio of the number of estimated eligibles to the number of completed surveys (weight=EE/E).
Once weights were assigned to each completed survey, the means of variables of interest were calculated based on unweighted and weighted data. The following is a summary of the findings for the variables 6A-I, 7A-K, 8A-D, 9A-I, and 11A-I.
TRANSPORTATION REVIEWERS
Difference between weighted and unweighted means ranges from .00 to .02
Average difference between weighted and unweighted means=.01
RESOURCE REVIEWERS
Difference between weighted and unweighted means ranges from .00 to .06
Average difference between weighted and unweighted means=.03
TRANSPORTATION MANAGERS
Difference between weighted and unweighted means ranges from .01 to .04
Average difference between weighted and unweighted means=.02
RESOURCE MANAGERS
Difference between weighted and unweighted means ranges from .00 to .02
Average difference between weighted and unweighted means=.01
The difference between the weighted and unweighted estimates (means) across the four groups was relatively small and hence it was not considered necessary to weight the 2006 survey data. It is also important to note that weighting was never used in the past for this particular survey.
Data Collection
In 2003, the main method of data collection was via phone. In fact, 92% of data in 2003 was collected over the phone using Gallup&146;s Executive Interviewing staff with only 7.6% using the Web. In 2006, overall 47% of responses came over the phone while 53% were through the Web. As the table below shows, these responses changed somewhat by region and by category (transportation vs. resource):
RESOURCE COMPLETIONS
|
|
Percentage of Phone Completes |
Percentage of Web Completes |
|---|---|---|
|
Region 1 |
68% |
85% |
|
Region 2 |
85% |
15% |
|
Region 3 |
36% |
64% |
|
Region 4 |
58% |
42% |
|
Region 5 |
47% |
53% |
|
Region 6 |
53% |
47% |
|
Region 7 |
44% |
56% |
|
Region 8 |
56% |
44% |
|
Region 9 |
62% |
38% |
|
Region 10 |
52% |
48% |
|
Total |
55% |
45% |
TRANSPORTATION COMPLETIONS
|
|
Percentage of Phone Completes |
Percentage of Web Completes |
|---|---|---|
|
Region 1 |
60% |
40% |
|
Region 2 |
63% |
37% |
|
Region 3 |
24% |
76% |
|
Region 4 |
27% |
73% |
|
Region 5 |
64% |
36% |
|
Region 6 |
37% |
63% |
|
Region 7 |
47% |
53% |
|
Region 8 |
45% |
55% |
|
Region 9 |
14% |
86% |
|
Region 10 |
33% |
67% |
|
Total |
41% |
59% |
Pre-Notification Letters
All potential respondents were sent pre-notification letters using e-mail addresses. Contained within the e-mail was a link to the survey and the respondent&146;s access code. The pre-notification letter in 2006 was signed by a Gallup project director and is attached to this report. The pre-notification letter was sent by Gallup on July 10, 2006. Throughout the interviewing process, as new potential respondents were identified, prenotes were continually sent out and names were added to the master list of potential respondents.
Potential respondents were sent follow-up reminder e-mails four times after the initial e-mail was sent asking for participation. The four reminder e-mails were sent on July 25, Aug.21, Sept. 12, and Nov. 14, 2006 to any respondents that had not completed a Web or a phone survey on those dates.
Phone Surveying
Phone surveying commenced on Aug. 1, 2006 with the Executive Interviewing staff calling only those respondents who had not completed a survey using the Web. Surveying via the Web continued until Nov. 13, 2006.
Five interviewers were selected to conduct the interviewers from the Gallup interviewing staff. All were chosen based upon their prior interviewing experience with this survey in 2003. Training occurred at the Lincoln, Nebraska location where interviewers were again instructed on the survey instrument, on differences from 2003, and briefed on the snowball sampling technique.
Follow-Up Procedures
The phone surveys used a 7x7 call design. Seven attempts were made to contact an eligible respondent and an additional seven attempts were made to interview that person. All calls were outbound with an 800 number left on answering machines to call Gallup back at the respondent&146;s convenience. If the interviewer discovered a number was either disconnected or ineligible, the number was returned to the Gallup recruiter and the number checked for accuracy. If there was still a problem, the Gallup recruiter would contact the agency for another phone number or another contact.
The final completion rates for this study are listed below in the table. Final completion rates were 61% for transportation agencies and 63% for resource agencies.
FINAL COMPLETION RATES
|
|
Transportation |
Resource |
|---|---|---|
|
Completes Phone |
378 |
373 |
|
Completes Web |
551 |
306 |
|
Total Completes |
929 |
679 |
|
Screen Failures/Ineligibles |
17 |
349 |
|
Screen Failures/Don't Know/Refused |
96 |
65 |
|
Call Back/Answering Machine |
362 |
604 |
|
Other |
175 |
359 |
|
Refusal |
21 |
55 |
|
Disconnected #'s |
21 |
80 |
|
less web completes |
-551 |
-306 |
|
Total Phone Contacts |
1070 |
1885 |
Respondents
Respondents from the surveys held a variety of professions. On the resource side, these included:
· Chief Environmental Officer
· Director
· Director of Planning
· District Engineer
· Division Chief
· Engineer
· Environmental Assessment Manager
· Transportation Specialist
· Archeologist
· Environmental Planner
· Biologist
· Environmental Specialist
· Environmental Analyst
· Environmental Program Manager
· Transportation Planner
· Transportation Engineer
· Environmental Engineer
· Engineering Manager/Supervisor
· Landscape Architect
· Preservation Specialist
· Fish and Wildlife Biologist
· Environmental Scientist
· Scientists
· Ecologists
· FHWA/transportation Liaison
Among the transportation professions, the list included:
· Design Engineer
· Director
· District Engineer
· Division Chief
· Engineer
· Environmental Assessment Manager
· Environmental Engineer
· Transportation Specialist
· Civil Engineer (Hydraulics/Structural Engineer)
· Operations Engineer
· Transportation Planner
· Transportation Engineer
· Project Development/Planning Engineer
· Construction Engineer
Who was Eligible to Participate?
To qualify for participate in this survey, respondents had to meet the following conditions:
1. A staff-level individual who works directly with NEPA-related documents, reviews or comments on NEPA-related documents, or somehow is directly related with the completion of these documents OR is a manager at either a resource or transportation agency who reviews the NEPA documents upon completion.
2. Respondent had to be working with NEPA in some capacity for at least two years.
3. Respondent had to have completed an Environmental Impact Statement (EIS), Categorical Exclusion (CE), Environmental Assessment (EA), or Programmatic Agreement (PA)* in the past six months.
*Note: No PA analysis was conducted due to insufficient sample size.
Eligible respondents were asked to choose a project that they completed within the selected time frame about which they were comfortable speaking and about which they had the most interaction with other sister agencies. Respondents then listed agencies that they worked with during the project and Gallup randomly selected an agency for the interview. That project and then the randomly selected agency were queried during the interview.
Sample Sizes
The following sample sizes apply to all data presented in this report. Because not all respondents answer all questions, when the sample size drops below 5, responses are not show either in the scorecards or in this report.
Region Managers Sample Size
|
|
Sample Size Resource |
Sample Size Transportation |
|---|---|---|
|
Region 1 - Managers |
33 |
38 |
|
Region 2 - Managers |
24 |
44 |
|
Region 3 - Managers |
29 |
31 |
|
Region 4 - Managers |
37 |
40 |
|
Region 5 - Managers |
39 |
48 |
|
Region 6 - Managers |
35 |
30 |
|
Region 7 - Managers |
16 |
28 |
|
Region 8 - Managers |
26 |
53 |
|
Region 9 - Managers |
44 |
36 |
|
Region 10 - Managers |
44 |
37 |
|
Total |
55% |
45% |
Region Reviewers Sample Size
|
|
Sample Size Resource |
Sample Size Transportation |
|---|---|---|
|
Region 1 - Reviewers |
35 |
56 |
|
Region 2 - Reviewers |
17 |
38 |
|
Region 3 - Reviewers |
49 |
56 |
|
Region 4 - Reviewers |
42 |
70 |
|
Region 5 - Reviewers |
37 |
52 |
|
Region 6 - Reviewers |
42 |
67 |
|
Region 7 - Reviewers |
27 |
23 |
|
Region 8 - Reviewers |
28 |
55 |
|
Region 9 - Reviewers |
34 |
67 |
|
Region 10 - Reviewers |
41 |
60 |
|
Total |
352 |
544 |
Marketing Research: Methodological Foundations. p.582. Sixth Edition. Churchill, Gilbert, A. The Dryden Press: Florida. 1995.