Highway Safety Improvement Program–Assessment Toolbox

Program Review

In general, a program review is a thorough analysis of key program components and the processes employed by the agency in managing the program. The reviews are conducted to 1) ensure compliance with Federal requirements; 2) identify opportunities for greater efficiencies and improvements to the program; or 3) identify exemplary practices. They can be referred to, or are known as, program improvement reviews, program assessments, program/product evaluations, or continuous process improvement initiatives.

A program review is an effective means to conduct a periodic evaluation of the overall HSIP, a particular element of the HSIP (i.e., SHSP, State HSIP, and RHCP) or to focus on a process within the HSIP (i.e., planning, implementation, and evaluation). Sample program review topics are included in the Appendix.

The Data portion of this assessment tool will focus on how data is utilized to support the various programs under the HSIP. An assessment of a State's crash data can be accomplished through the Crash Data Improvement Program (CDIP). An assessment of a State's roadway data can be accomplished through the Roadway Data Improvement Program (RDIP). Lastly, an assessment of your State's entire traffic records system can be achieved through a Traffic Records Assessment sponsored by the National Highway Traffic Safety Administration.

The targeted area for a program review is often identified through the Division Office's risk assessment process, in consultation with the SDOT, based on the stewardship and oversight agreement. Safety specialists might consider using the HSIP Self-Assessment Tool (as described in the previous section), to feed the risk assessment process. A program review may be warranted if the discussion surrounding a particular question indicates:

• Opportunities for improvement in a particular area.
• This is not how the SDOT currently does business.
• There has been no change in the current process for a number of years.
• A new process has been established recently.

The following are examples of questions that a HSIP Program Review may help to answer:

• Is the State seeing a positive impact from the implementation of the HSIP?
• Is the HSIP in compliance with the law and regulation?
• Is the State using the best project identification process given the available data?
• Is the HSIP truly data driven?
• Does the HSIP address all public roads?
• Can areas for improvement be identified?
• Are there noteworthy practices that would benefit HSIP peers?
• Are the HSIP evaluation results feeding back into the planning element of the HSIP and affecting department policies and design standards?
• Does the HSIP consider an appropriate balance between spot and systemic safety projects?
• Why are HSIP obligation rates lower than that of other Federal-aid program areas?

The examples provided below further illustrate scenarios for which a HSIP program review may be beneficial.

Steps for Conducting a Program Review

Typical steps for conducting a program review are as follows:

• Assemble a multi-disciplinary team. [A multidisciplinary team would include representatives from all units, teams, or departments responsible for carrying out the HSIP.]
• Develop a review plan. [The review plan should consist of a purpose, scope, expected results, information needed, gaps in information and possible sources, team members and roles, schedules, and resources.]
• Conduct review.
• Analyze and interpret results.
• Develop inferences, recommendations, and lessons learned.
• Prioritize recommendations and lessons learned.
• Present the findings from the review.
• Apply recommendations and lessons learned.

The Work Zone Process Review Toolbox outlines the Steps for a Review (as noted above) in the "ABCs" of Process Reviews and provides additional explanatory details for each step of the program review process. While there are references to work zone process reviews throughout the document, the steps are generic enough to apply to all program areas, including the HSIP. Follow-up and monitoring are essential to the success of program reviews. Follow-up and monitoring may be in the form of regularly scheduled status meetings (i.e., quarterly, biannual) or presentations at appropriate meetings of task forces, managerial groups, etc.

Resources

The National Highway Institute hosts several courses related to program reviews including Conducting Effective Program Reviews (310120), Introduction to Data Analysis (310122) and Writing Effective Program Review Reports: Moving People to Action (310119). These workshops provide training and hands on assistance in the methodology and tools available for conducting successful reviews.

A sample set of questions is included in the Appendix. These questions can be used to support the information gathering process (as identified in Step 2 and obtained in Step 3 of the review process), whether in the format of an interview or a questionnaire. These questions can be used as a starting point; however, they should be modified to address the intent of the program review and the target audience. They are organized by program area (General, SHSP, RHCP, and Program of Highway Safety Improvement Projects) and within each program by process (i.e., planning, implementation, evaluation). Examples of additional information resources that can support the review process are included in the Appendix.

Many Divisions have conducted HSIP-related program reviews, some of which are available in the Program Review Library [available to FHWA personnel only]. It may be beneficial to review some of the HSIP-related program review reports to get a feel for what has been done in other States. In many instances, the report outlines the process used to conduct the review as well, which could prove useful in planning your review. The Program Review Library also contains a program review report template and writing and style guide that should be used to document your review efforts.

Noteworthy practices can be used as the basis for the recommendations set forth in Step 5 of the review process. FHWA hosts a roadway safety noteworthy practices database that can be used to explore other State HSIP practices. If a State is interested in further exploring how another State does business in a particular area, it may be beneficial to host an HSIP-related peer review (as described in the next section).