Ongoing evaluation of program outcomes is essential to assure that the program is providing the desired benefits. Here we will provide an overall evaluation approach that using a framework that is becoming increasingly popular in evaluating dissemination of evidence based practices.
- Implementation fidelity
The extent of the program’s reach can be determined by tracking the numbers of clients identified as depressed through the CARE tool or other depression screening system that receive treatment. As a program for treatment of depression, the primary outcome of effectiveness is the change in the PHQ-9 depression score from baseline to the final assessment. However, other, secondary outcomes are very important to track and include changes in levels of pleasant events scheduling, and physical and social activation. Participant satisfaction and in particular, the qualitative statements they make about how they have benefited by participating in the program are powerful parts of the evaluation. PEARLS was not designed to save health care dollars but the research supporting its effectiveness suggest that it may reduce overall hospitalization rates and expenditures for participants. Thus, depending on the size of one’s organization or the evaluation perspective (e.g. state wide) it may be important to evaluate hospitalization data. Adoption of the PEARLS program refers to how well the program spreads throughout the organization. For example, are clients of all case managers likely to be included or do only a limited number of case managers adopt or use the program? Implementation fidelity is determined by evaluating how closely the program adheres to the evidence based practice model. Currently, there is not a quantitative measure of PEARLS program fidelity. However it is possible to determine how well the program practices as described in this toolkit are carried out on a day to day basis. For example the organization can determine, do the clients that are enrolled clearly have minor depression or dysthymia or has the program moved to enrolling other types of participants. Are the PHQ-9 scores obtained consistently in every PEARLS sessions? Are there processes in place for psychiatric supervision of the counselors? Maintenance of the PEARLS program refers to the sustainability over time of the program and can be determined by tracking the years, numbers of participants enrolled annually across the history of the program.
Program Evaluation is discussed further in the PEARLS Toolkit