CHECK: Evaluating Program Effectiveness
The next step in determining the effectiveness of an operator’s engagement program is the evaluation of information collected on process and program implementation. Operators are required in RP 1185 to have a process for evaluating the effectiveness of their program that also demonstrates how they are connecting their program to the PDCA cycle.
Operators are encouraged to use the minimum program documents as the basis for their evaluation process. Operators are also encouraged to incorporate both quantitative and qualitative data collection methods, which include seeking stakeholder feedback on engagement activities.
Quantitative data will help identify overall program performance trends, but qualitative methods may be needed to determine if a particular point or method of engagement satisfied stakeholders.
For example, tracking the observations, comments, concerns, or information requests that stakeholders submit through established contact methods will allow an operator to determine if the established contact methods are being used and if they are responding to stakeholder requests in a timely manner.
However, conversations with stakeholders are likely required to understand if those contact methods have been clearly communicated and to confirm that all stakeholders can access and use them. Similarly, an operator may need follow-up conversations to determine if their responses to stakeholder requests are meeting the mark – from both the stakeholder’s point of view and the operator’s view of process implementation.