Merge Ahead: Integrating Process and Impact Evaluations

Merge Ahead: Integrating Process and Impact Evaluations

Patrick Hewlett and Sue Haselhorst for Zondits, August 27, 2014

Presented at ACEEE Summer Study 2014

Traditionally, process and impact evaluations are conducted independently, under separate contracts and in separate timeframes. Frequently, there is little opportunity for communication and sharing between the two teams. In this environment recommendations can be vague and/or disconnected from program realities. In the most extreme cases, the evaluation can yield unrelated and even contradictory recommendations. For example, for a large small business direct install program in one jurisdiction, the process team recommended increasing outreach to franchises for more efficient sales, while the impact team simultaneously recommended staying away from franchises because of high free ridership.

Merging the process and impact evaluations can provide a host of beneficial synergies and produce avenues for internal collaboration and cross-pollination that should yield more informed and robust recommendations. This paper will describe how an integrated approach was applied to one large program in an effort to enhance the overall outcome. From a review of the methods for integrating the two evaluations to a critical look at the pluses and minuses of integration, this paper will discuss the following aspects of the merge:

  • Planning – Alignment of objectives; priorities for survey questions; data sharing; task schedule benefits
  • Data collection – Impact team providing hard data to process team; process team providing soft data to the impact team; coordinated or combined telephone surveys
  • Recommendations – Looking for overlap; using impact results to assign a value to a recommendation and using process-team finds to pinpoint how that recommendation might be implemented

[slideshare id=38409325&doc=integratingprocessimpactevals-140827075408-phpapp01]