Abstract: Publication date: Available online 8 November 2019Source: Evaluation and Program PlanningAuthor(s): James J. Annesi Camp leadership programs have the potential of improving important psychosocial factors in teens who might eventually become counselors. Accepted behavioral theory might increase understandings of those changes so program training architectures can be improved. As part of larger-scale longitudinal research, participants (N = 129; 64% female, Mage = 14.6 years, SD = 0.6) were selected by senior camp administrators for a structured leadership training program that required in-person conferences, viewing training videos every 2 weeks, obtaining ongoing mentoring from senior camp staff, and attending several weeks of summer camp where further training was provided. Validated surveys of proposed dimensions of self-control and interpersonal functioning were administered twice over 6 months to assess training program-associated changes. There were significant improvements in total self-control, self-control specifically related to coping, and family-related functioning. Although no demographic factor was associated with score change, baseline score was a significant predictor across measures. After controlling for baseline scores, stepwise multiple regression analyses indicated that higher self-control scores (i.e., planful behavior, coping efficacy, affective thought management) significantly predicted improvements in general, family, and peer-related functioning. Theory-based relationships between changes in self-control and interpersonal functioning enabled the identification of training program areas that might be targeted for improvement.
Abstract: Publication date: Available online 7 November 2019Source: Evaluation and Program PlanningAuthor(s): Joshua Brisson, Igor Pekelny, Michael Ungar This article reflects on common challenges and lessons learned during the evaluation of gang prevention programs based on case studies of three federally funded Canadian programs. Elements of evaluation design, implementation, data analysis and reporting of results are discussed. More specifically, the article highlights issues that occur when evaluating community projects focused on preventing extreme risks for violence and the complexity of working in potentially dangerous and/or unstable work environments. Topics covered include the problem with quasi-experimental designs, model fidelity adherence, program documentation, client recruitment and retention, and data collection. Recommendations are provided to improve evaluations of youth gang prevention programs and similar community-based interventions that focus on the prevention of youth violence.
Abstract: Publication date: Available online 6 November 2019Source: Evaluation and Program PlanningAuthor(s): Wenhui Pan, Pengwei Zhao, Chunxiu Qin, Xianfeng Ding The paper focuses on how to improve academic output of a granted fund when the adding of new members changes principal investigator’s network position. This objective is refined by exploring how new members affect the relationship between principal investigator’s network position and the academic output of granted funds, and whether this effect is similar in high-output and low-output collaboration networks. New members are divided into two groups, namely, international collaboration and inbound mobility. Using negative binomial regression on research funds supported by the National Natural Science Foundation of China, the results indicate that that new members indeed have moderating effects on the relationship between principal investigator’s network position and the academic output of granted funds. Moreover, we find that this moderating effect in high-output collaboration networks is larger than that in low-output collaboration networks. Our findings provide practical implications for the decision makers to design funding planning and for the principal investigators to improve academic output.
Abstract: Publication date: Available online 6 November 2019Source: Evaluation and Program PlanningAuthor(s): M.E. Menconi, S. Tasso, M. Santinelli, D. Grohmann Urban parks are often designed and renewed using standardized models that fail to incorporate the different functions and services required by the local communities served. Furthermore, when the citizens are involved in co-design, the participation processes are seldom fully inclusive, and the results are not representative of the entire local community. The paper starts from these considerations for developing an innovative participatory design method to renew urban parks. This method simultaneously uses multiple and alternative participation tools for adapting to the different technical skills of the citizens and to their inclinations. Furthermore, it entails the analysis and interpretation of the expectations of the local community in light of the urban green network. The method uses the conceptual framework of the SWOT analysis and the Trochim's concept mapping for developing a card game playable face to face or online. The results of the card game are processed using the statistical analysis to identify the main design themes (the clusters), and to split the participants’ suggestions into different design alternatives. Successively, the participants vote these alternatives for selecting the renewal design respondent to the local community's needs.
Abstract: Publication date: Available online 3 November 2019Source: Evaluation and Program PlanningAuthor(s): LaShawn Glasgow, Elizabeth Adams, Lucia Rojas Smith, Jeanette Renaud
Abstract: Publication date: Available online 2 November 2019Source: Evaluation and Program PlanningAuthor(s): Rocio Rodriguez, Göran Svensson, Greg Wood PurposeThe purpose of this study is to examine the previous and the current efforts and the priorities of sustainability initiatives in the public sector.Design/Methodology/approachThis study has been conducted in one industry, overcoming contextual bias as a judgmental sampling was used to select the public hospitals that were studied. Only knowledgeable key informants were approached and used.FindingsThe examination of the previous and the current efforts and the priorities of the sustainability initiatives revealed in this study indicate the existence of different trends in the public hospitals studied.Research limitations/implicationsThe results report the key trends that were disclosed in the public hospitals studied in their efforts towards sustainability. In particular, the results show that there appears to be a lack of guidelines and homogeneity in sustainability planning in public hospitals in Spain, accompanied by the near universal absence of the evaluation phase in respect to the outcomes of the sustainability initiatives that have been put in place in these organizations.Managerial ImplicationsThe multi-dimensional factors of sustainability initiatives provide managerial guidance to assess the previous and the current efforts and priorities. These factors also provide organizational guidance to assess the trends of an organization through time.Originality/ValueThis study contributes to a selection of factors regarding the previous and the current efforts and the priorities of sustainability initiatives applicable in public hospitals. It provides a multi-dimensional framework of factors that can be used in order to describe sustainability trends.
Abstract: Publication date: Available online 31 October 2019Source: Evaluation and Program PlanningAuthor(s): Deirdre Dingman, Jingwei Wu, Heather M. Murphy In the United States, lack of proper handwashing is associated with respiratory and gastrointestinal illnesses. Interventions to improve handwashing practices have led to an increase in handwashing knowledge and behavior, and decreases in gastrointestinal illnesses. Most studies have evaluated their interventions in the context of reported handwashing rates by observation, reduction of illnesses, as well as reduced absences, however none of these studies examined handwashing quality or knowledge as outcome measures. The objective of this paper is to present the results from a handwashing program with a special focus on the evaluation methods.A pre-post design was used to evaluate a handwashing program that took place in two pre-schools the northeast of the United States. The program utilized a black light technology to demonstrate to children the importance of good quality. The evaluation consisted of assessing knowledge and quality of handwashing using a linear puzzle and individual handwashing observation, respectively.Students from both schools improved on both knowledge and quality over time (p-values 0.071 and
Abstract: Publication date: Available online 22 October 2019Source: Evaluation and Program PlanningAuthor(s): Chunyan Liu, Leiheng Wang In order to achieve “gender mainstreaming” which is a global strategy to promote the development of women, it has in recent years been a major task for the All-China Women's Federation (ACWF) to improve the gender awareness and ability of policy makers and implementers. Thus, researchers should constantly propose and evaluate various possible attempts to improve gender awareness and the ability of individuals. There are few empirical studies on the assessment of gender awareness and the improvement of the ability of staff who have been trained and educated. Research studies focusing on the promotion of gender-sensitive awareness and ability of staff by participating in programs are even rarer.This study presents a specific case, namely the evaluation of promoting social service workers’ gender awareness and ability within the implementation of the “3861 My Home” program. Based on this case, this study examines how to promote gender awareness and ability of social service workers through the implementation of the service program of Women's Federation, and also assesses the quality of the results of this implementation. Moreover, this study also contributes to the literature pertaining to assessing the development of social service workers’ gender awareness and ability. In addition, the study provides an option for enhancing the gender awareness and ability of social service workers in addition to training: The participatory learning or action pertaining to gender awareness and ability based on the Women's Federation programs are able to effectively improve the gender awareness and ability of staff.
Abstract: Publication date: Available online 22 October 2019Source: Evaluation and Program PlanningAuthor(s): Traci H. Abraham, Tisha L. Deen, Michelle Hamilton, Gala True, Marie T. O’Neil, Jessica Blanchard, Madeline Uddo Despite widespread availability of yoga in the Veterans Health Administration (VA), it remains unclear how to best evaluate yoga programs. This is particularly problematic for programs aimed at veterans with mental health concerns, as evaluation typically focuses narrowly upon mental health symptom severity, even though program participants may have other health-related priorities. We analyzed responses to free-text questions on 237 surveys completed by veterans with mental health concerns enrolled in a yoga program at six VA clinics in Louisiana to characterize veteran participants’ experiences with yoga. Qualitative analysis resulted in 15 domains reflecting veterans’ individual health-related values and priorities. We use results to illustrate the potential for analysis of free-text responses to reveal valuable insights into patient experiences, demonstrating how these data can inform patient-centered program evaluation. The approach we present is more accessible to those responsible for decision-making about local programs than conventional methods of analyzing qualitive evaluation data.
Abstract: Publication date: Available online 18 October 2019Source: Evaluation and Program PlanningAuthor(s): Rachel Claire Mitchell, Glenn D. Israel, David C. Diehl, Sebastian Galindo-Gonzalez The Southeastern Coastal Center for Agricultural Health and Safety (SCCAHS) is one of many newly-funded federal research centers, housing five multidisciplinary research projects and seven pilot projects, and serving a multi-state region. In the early stages of such a complex project, with multiple teams separated by geography and disciplines, the evaluation program has been integral in connecting internal and external stakeholders at the center and project levels. We used a developmental evaluation (DE) framework to respond to the complex political environment surrounding agricultural health and safety in the southeast; to engage external stakeholders in guiding the center’s research and outreach trajectories; to support center research teams in a co-creation process to develop logic models and tailored indicators; and to provide timely and feedback within the center to address communications gaps identified by the evaluation program. By using DE principles to shape monitoring and evaluation approaches, our evaluation program has adapted to the dynamic circumstances presented as our center’s progress has been translated from a plan in a grant proposal to implementation.
Abstract: Publication date: Available online 9 October 2019Source: Evaluation and Program PlanningAuthor(s): Abigail M Hatcher, Ruari McBride, Dumisani Rebombo, Shehnaz Munshi, Mzwakhe Khumalo, Nicola Christofides Intimate partner violence (IPV) is experienced by one-third of women globally, yet few programs attempt to shift men’s IPV perpetration. Community mobilization is a potential strategy for reducing men’s IPV perpetration, but this has rarely been examined globally. We conducted a mixed-methods process evaluation alongside a trial testing community mobilization in peri-urban South Africa. We used in-depth interviews (n=114), participant observation (160 hours), and monitoring and evaluation data to assess program delivery. Qualitative data (verbatim transcripts and observation notes) were managed in Dedoose using thematic coding and quantitative data were descriptively analyzed using Stata13. We learned that outreach elements of community mobilization were implemented with high fidelity, but that critical reflection and local advocacy were difficult to achieve. The context of a peri-urban settlement (characterized by poor infrastructure, migrancy, low education, social marginalization, and high levels of violence) severely limited intervention delivery, as did lack of institutional support for staff and activist volunteers. That community mobilization was poorly implemented may explain null trial findings; in the larger trial, the intervention failed to measurably reduce men’s IPV perpetration. Designing community mobilization for resource-constrained settings may require additional financial, infrastructural, organizational, or political support to effectively engage community members and reduce IPV.
Abstract: Publication date: Available online 5 October 2019Source: Evaluation and Program PlanningAuthor(s): Saul Ngarava, Abbyssinia Mushunje, Petronella Chaminuka The objective of the study was to develop an index which circumvents the need for quantitative measures in evaluating Livestock Development Programmes (LDPs). This was done through a conceptual framework which integrates indexing and the Sustainable Livelihoods Framework (SLF) to attain qualitative benefits. The study utilised a cross-sectional survey of Kaonafatso ya Dikgomo (KyD) Scheme participants in KwaZulu-Natal, Eastern Cape and Limpopo Provinces, South Africa, as a case. Data were collected through a questionnaire. Multiple sampling methods were used to identify 164 respondents. The results showed low benefits as exhibited by an index of 0.26. The Scheme showed highest benefits through social capital, followed by human, natural, financial and physical capital, respectively. The study concluded that concentrating on income proxies through convectional cost-benefit analysis (CBA) tends to mask the qualitative benefits exhibited by livestock improvement programmes. The theoretical lessons learnt included the exhibition and possibility of measuring qualitative benefits through indexing. Recommendations include further inquiry and refining the construction of the index to circumvent its subjectivity.
Abstract: Publication date: Available online 3 October 2019Source: Evaluation and Program PlanningAuthor(s): Aduragbemi Banke-Thomas, Sonja Nieuwenhuis, Adesoji Ologun, Gordon Mortimore, Martin Mpakateni In recent times, there has been an increasing drive to demonstrate value for money (VfM) for investments made in public health globally. However, there is limited information on practical insights and best practices that have helped implementing organisations to successfully embed VfM in practice for programming and evaluation. In this article, we discuss strengths and weaknesses of approaches that been used and insights on best practices to manage for, demonstrate, and compare VfM, using a health pooled fund programme implemented in conflict-affected South Sudan as case study supported by evidence reported in the literature while critiquing adequacy of the available approaches in this setting. An expanded and iterative process framework to guide VfM embedding for health programming and evaluation is then proposed. In doing so, this article provides a very relevant one-stop source for critical insight into how to embed VfM in practice. Uptake and scale-up of the proposed framework can be essential in improving VfM and aid effectiveness which will ultimately contribute to progress towards achieving the Sustainable Development Goals by 2030.
Abstract: Publication date: Available online 30 September 2019Source: Evaluation and Program PlanningAuthor(s): Rachel Palmén, Evanthia Kalpazidou Schmidt
Abstract: Publication date: Available online 30 September 2019Source: Evaluation and Program PlanningAuthor(s): Kyoo-Man Ha It is not widely known that quite a few researchers are faced with difficulties in using various resources of disaster management research in Korea. The article aims to assess how rigorously the Korean field of disaster management research resources has been managed or how it can be improved for the ultimate goal of disaster management. Descriptive content analysis has been used as the major methodology by referring to the Johari window. In doing so, electronic research resources have been systematically compared with integrated research resources via the perspective of Korean-speaking researchers and that of English-speaking researchers. The conclusion is that two researchers have to be integrated with all four research resources (open, blind, hidden, and unknown resources) by implementing assigned responsibilities as well as freely asking questions. Ultimately, this will be conducive to reducing down the impacts of disaster in Korea.
Abstract: Publication date: Available online 16 September 2019Source: Evaluation and Program PlanningAuthor(s): Holly McClain Burke, Samuel Field, Lázaro González-Calvo, Margaret A. Eichleay, Troy D. Moon PurposeOur goal was to determine whether an economic and social empowerment intervention implemented in Zambézia Province, Mozambique reduced girls’ vulnerability to HIV. We use this experience to discuss challenges of evaluating real-world interventions.MethodsTwo rounds of data were collected from 885 girls, 13-19 years, for this clustered, non-equivalent (two-stage) cohort trial. We used multi-level exact matching and difference-in-differences estimation to estimate intervention effects on two outcomes: girls’ knowledge of gender-based violence and school attendance.ResultsEstimates of two outcomes analysed indicated no statistically significant intervention effects. Preliminary analysis of data from the intervention group revealed this study was unable to obtain accurate measures for five outcomes related to HIV vulnerability.ConclusionsAlthough our study did not find evidence of impact on the a priori selected outcomes, we report on our experience implementing this robust methodologic design and describe how the challenges encountered in this program setting affected our ability to attain results. We recommend prospective evaluation designs with random allocation be accommodated early during planning. When not possible, quasi-experimental studies should collect data from large samples. To reduce measurement bias, biological endpoints such sexually transmitted infections should serve as primary outcomes for programs intending to reduce sexual behaviors.
Abstract: Publication date: Available online 12 September 2019Source: Evaluation and Program PlanningAuthor(s): Andrew Pleasant, Catina O’Leary, Richard H. Carmona This case study attempts to illustrate and address in-depth the issues surrounding the collection, analysis, and application of formative research findings to program development and implementation. We provide an in-depth case study of tailoring a program for the residents of Berkshire County, Massachusetts. The formative research process includes collection and analysis of secondary data sources, extensive in-person interviews with community leaders, and in-depth focus groups with members of the population of interest. Findings from the formative research are then applied to tailoring the program materials and presentations and the training of the integrative team of health professionals that offer the program. Distinct components of program are tailored to the realities of the social, cultural, historical, and health and medical contexts in each community while other components of the program are tailored to individual participants. Overall, we believe this case study fully illustrates the utility of formative research in tailoring evidenced-based programs to increase program relevance and positive outcomes while maintaining fidelity to a program’s learning objectives and evaluation. We hope this in-depth account with specific examples proves useful as a guide to others when designing and conducting formative research to tailor health and medical interventions to the audience.
Abstract: Publication date: Available online 12 September 2019Source: Evaluation and Program PlanningAuthor(s): Ricard Esparza Masana, Tatiana Fernández Our work analyses the implications of a smart specialisation strategy (S3) monitoring model based on a system that considers three main axes linked to the logic of the strategies: output/results, smart specialisation process, and a learning component, on which we build our main approach. This strategic learning is based on the logic that S3 presents a new paradigm within the innovation policy scope that requires the inclusion of monitoring schemes that integrate the assessment of the dynamic process under which both public administration and other innovation stakeholders learn and improve their activities (within and without the strategy) thanks to the learning gathered throughout S3 (and its instruments and projects) implementation. The article also discusses this logic within the current and the next programming period for S3, and the role monitoring has or will have, while analysing the opportunities and challenges it presents.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): James Onken, Richard Aragon, Anna Maria Calcagno This article examines the geographic distribution of funding for the U.S. Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs sponsored by the National Institute of General Medical Sciences (NIGMS). Despite a significant investment in SBIR/STTR and an interest in increasing geographic diversity in the institute’s research portfolio, there has not been an assessment of the distribution of NIGMS’s SBIR/STTR funding, outcomes associated with that investment, and relationships between the two. The geographic distribution of NIGMS’ SBIR/STTR funding was highly concentrated in a small number of states, with a high correlation between each state’s funding and its number of small scientific research and development businesses. Affiliation with a major research university was correlated with several measures of innovation and firm success. Our findings are consistent with earlier research showing that economic activity in research and development and research output tend to cluster in geographic regions where knowledge can be generated and shared more efficiently. These findings lend support to an investment strategy for small business research and development that creates networks between major research universities and small businesses.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Chokri Kooli This paper aims to evaluate the role and effects of national accreditation practices in the improvement and development of the Omani private higher educational institutions in term of Governance and Management. A documental collection, revision and analysis was performed in order to enhance our understandings of the impact of quality audit in terms of Governance and Management. The study focused on analysing the quality audit reports of 25 assessed Omani HEIs. It covered published reports of all private institutions who already completed the first stage of national accreditation process. From one side, the data analysis showed us that the majority of the Omani private higher education institutions operate their activities without having a clear strategic direction. Also, they don’t have performant governance and management systems. From another side, a progress was observed in terms of institutional Affiliations for Programs and Quality Assurance, Student’s grievance processes and Health and Safety management. Among other things, the findings revealed that the Omani HEIs perceive quality assessment programs as strategical imposed tool made under the constraint of accountability, rather than an effective mechanism of development.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Bader Alhafi Alotaibi, Edgar Yoder, Mark A. Brennan, Hazem S. Kassem The purpose of this study was to investigate the needs of extension agents for training on organic agriculture (OA) in the Riyadh Region. Fifty-five percent (n = 69) of extension agents were selected to collect data through an electronic survey. Results indicated that extension agents in the Riyadh Region cited on job training as a main source for acquiring knowledge in OA. Agents expressed interest in receiving training in OA in general, OA marketing, and transitioning to OA. The most useful resources were field days at organic farms, OA training sessions at colleges, and attending OA trial plots. There were no differences in the level of interest in training; however, there were significant differences between individuals in terms of their experience in OA, job responsibility for providing information related to OA, and area of specialization in their job. This study recommends developing professional training programs based on the current knowledge of extension agents.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Charlotte Skau Pawlowski, Tanja Schmidt, Jonas Vestergaard Nielsen, Jens Troelsen, Jasper Schipperijn Knowledge on how to improve public open spaces in deprived neighbourhoods to increase active living among children is scarce and comprehensively evaluated public open space interventions are needed. Firstly, the aim was to explore if involving 39 local fifth-grade children (10–11 years old) from a deprived neighbourhood in creating playable installations in a public open space influenced their use of this space. Secondly, we wanted to explore if the Reach, Effectiveness, Adoption, Implementation and Maintenance framework (RE-AIM) was useful for evaluating different stages in the intervention project. RE-AIM was applied through a convergent mixed-methods triangulation design using survey, accelerometer, GPS, and interviews as data sources. Effectiveness outcomes revealed that on average the space was used less by the 39 children after the intervention. The implementation and maintenance dimensions revealed aspects of why most children involved in the project did not use the space after intervention. The evaluation cast light on children’s perceptions of their role, and importance of maintenance when the intervention was completed. In future, all dimensions of built environmental projects would benefit from being planned and evaluated in a collaboration with all project partners using an evaluation framework integrated and applied from the beginning of the project.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Madjid Tavana, Ghasem Khosrojerdi, Hassan Mina, Amirah Rahman The primary goal in project portfolio management is to select and manage the optimal set of projects that contribute the maximum in business value. However, selecting Information Technology (IT) projects is a difficult task due to the complexities and uncertainties inherent in the strategic-operational nature of the process, and the existence of both quantitative and qualitative criteria. We propose a two-stage process to select an optimal project portfolio with the aim of maximizing project benefits and minimizing project risks. We construct a two-stage hybrid mathematical programming model by integrating Fuzzy Analytic Hierarchy Process (FAHP) with Fuzzy Inference System (FIS). This hybrid framework provides the ability to consider both the quantitative and qualitative criteria while considering budget constraints and project risks. We also present a real-world case study in the cybersecurity industry to exhibit the applicability and demonstrate the efficacy of our proposed method.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): James Onken, Andrew C. Miklos, Travis F. Dorsey, Richard Aragon, Anna Maria Calcagno Here, we report the results of an outcomes evaluation of the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) Programs at the National Institute of General Medical Sciences (NIGMS). Since the programs’ inception, assessments of the SBIR/STTR programs at several federal agencies have utilized surveys of former grantees as the primary source of data. Response rates have typically been low, making non-response bias a potential threat to the validity of some of these studies’ results. Meanwhile, the availability of large publicly-available datasets continues to grow and methods of text mining and linking databases continue to improve. By linking NIGMS grant funding records, U.S. Patent and Trademark Office data, and business intelligence databases, we explored innovation, commercialization and survival for recipients of NIGMS SBIR/STTR funding. In doing so, we were able to more completely assess several key outcomes of the NIGMS SBIR/STTR program. Our evaluation demonstrated that the NIGMS program performed above baseline expectations along all dimensions, and comparably to other federal agency SBIR/STTR grant programs. In addition, we show that the use of extant data increasingly is a viable, less expensive, and more reliable approach to gathering data for evaluation studies.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Stella Nordhagen, Jennifer Nielsen, Tom van Mourik, Erin Smith, Rolf Klemm Using agriculture to improve nutrition is an approach growing in popularity, with programs becoming increasingly complex and multisectoral. While there is an active line of research assessing the impacts of such programs, little has been written about the process of successfully implementing them. As such, this paper uses a multisectoral nutrition-sensitive agriculture program implemented in four African countries as a case study to address key challenges in and lessons learned from implementation. We highlight the overall flexibility of nutrition-sensitive agriculture but also the need to adapt certain aspects to the particular context, as well as the opportunities for cross-context learning (and the limits to this). Integrating rigorous evaluation into such complex programs and forging diverse cross-sectoral partnerships offer both rewards and challenges, upon which we reflect. Main lessons learned from the program include the importance of carefully sequencing interventions, retaining flexibility in implementation, allowing for considerable time for cross-sector integration and coordination, and considering community impacts when designing research.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Elizabeth Ruvalcaba, Jennifer Callaghan-Koru, Cynthia S. Rand, Michelle N. Eakin PurposeTo understand the role of Maryland Head Start (HS) programs in asthma care and identify resources and needs to improve health outcomes.MethodsA qualitative needs assessment was conducted with Maryland HS staff (n = 35) and parents/caregivers of enrolled children with asthma (n = 16) from all 14 grantee programs in Maryland. Focus group discussions and interviews addressed strengths and challenges in current asthma control and opportunities for integration of an asthma care program into HS services. Transcripts were thematically analyzed using a modified Framework approach.ResultsHS programs actively communicate with families about asthma management and facilitate communication between families and primary care providers (PCPs). Both HS staff and families reported a strong trusting relationship allowing HS staff to provide asthma management support. HS needs strong linkages with supportive services and PCP offices to engage families, address environmental triggers, and educate staff. While families across the state report interest in peer education on asthma through HS, there were some region-specific asthma care barriers for urban and rural programs.ConclusionThis needs assessment confirmed interest among all stakeholders in integrating asthma care through HS and identified communication strategies, supportive infrastructure, and addressing regional access to care as key elements for program design.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Freda N. Gonot-Schoupinsky, Gulcan Garip Health interventions exhibit three stages of maturity: early-, mid-, and late-stages. Early-stage interventions have innovative content necessitating evaluation; however existing evaluation frameworks omit constructs and guidelines relevant to this evaluation. Early-stage interventions require planning and evaluation that supports creating, testing, and exploring content to establish general feasibility and enable refinement for further testing, prior to randomised controlled trialling and wider dissemination. Feasibility, Reach-out, Acceptability, Maintenance, Efficacy, Implementation, Tailorability (FRAME-IT) was developed for a mixed methods feasibility study of a novel well-being intervention. FRAME-IT was conceived as a complementary framework to Reach, Efficacy, Adoption, Implementation, Maintenance (RE-AIM; Glasgow et al., 1999) which is better suited for mid- and late- stage interventions. FRAME-IT is proposed to support: (1) early-stage intervention planning and design, by guiding research focus and data sourcing strategy with relevant constructs; (2) comprehensive evaluation, by including constructs appropriate for early-stage interventions, i.e. feasibility, acceptability, and tailorability; (3) future intervention scalability, by including and adapting some of RE-AIM’s constructs to encourage a smoother translation of research into practice as interventions are scaled-up.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Evans Sakyi Boadu, Isioma Ile Participatory monitoring and evaluation (PM&E) approaches have been touted as an important concept for local participation in social interventions programmes. Utilizing a process analysis approaches, the PM&E data gathered through key informant interviews, formal surveys, and policy frameworks were analyzed. The study revealed three dominant power struggles between youth and programme implementers in three districts at the Greater Accra region, Ghana. The first and pervasive form of power dynamics involves the youth and programme implementers. The second involves a variety of arrangements with the government on one hand and implementers and youth on the other side. The third is who qualifies to be a beneficiary and for that matter participate in the PM&E. In these three forms of power struggles the paper reveals superior implementers control of who participate in the programme PM&E and at what stage in the process. The power imbalance between programme implementer and target beneficiaries impeded the level of beneficiaries’ participation in the PM&E.
Abstract: Publication date: December 2019Source: Evaluation and Program Planning, Volume 77Author(s): Holly McClain Burke, Catherine Packer, Lázaro González-Calvo, Kathleen Ridgeway, Rachel Lenzi, Ann F. Green, Troy D. Moon PurposeAn intervention including business training and health education was implemented in Mozambique, where girls are at elevated risk for acquiring HIV. As part of a mixed-methods evaluation, we describe perceived effects of the intervention on girls’ sexual behavior and school attendance.MethodsWe conducted 49 in-depth interviews (IDIs) with girl intervention participants (ages 13–19), 24 IDIs with heads of girls’ households, 36 IDIs with influential males identified by girls, and 12 focus group discussions with community members after the intervention ended and one year later.ResultsInformants said the primary intervention benefit was realized when girls had money to stay in or return to school and/or to buy necessities for themselves and their households—reducing their need for transactional or intergenerational sex. However, some girls did not make a profit and some businesses were not sustainable. Sometimes the intervention appeared to be implemented in a way to reinforce inequitable gender norms resulting in some girls feeling shame when they reengaged in risky sex after their businesses failed.ConclusionsEarning money enabled girls to potentially reduce their vulnerability to HIV. We offer recommendations for future multi-sector interventions, including the need to address potential harms in programs serving vulnerable girls.
Abstract: Publication date: Available online 11 September 2019Source: Evaluation and Program PlanningAuthor(s): Huey T. Chen, Liliana Morosanu, Kia Powell-Threets, Brad Lian, Nannette Turner
Abstract: Publication date: Available online 10 September 2019Source: Evaluation and Program PlanningAuthor(s): Allison M. Sweeney, Dawn K. Wilson, Asia Brown Self-Determination Theory proposes that some people are motivated to initiate physical activity by externally-controlled reasons (e.g., physical appearance, approval from others), whereas others feel compelled by more autonomous sources of motivation (e.g., enjoyment, personal importance). This study used qualitative methods to examine whether individual differences in autonomous motivation offers a useful framework for developing targeted intervention programs for African American women. Focus groups (k = 6) were conducted to examine how different levels of autonomous motivation for physical activity related to differences in physical activity barriers and facilitators among African American women (N = 31). Sessions were audiotaped, transcribed, and coded by independent raters (r = .70). QSR NVivo 11 was used to analyze data, and themes were identified separately for women with low, medium, or high autonomous motivation. Those with high autonomous motivation focused on themes of wanting novelty, excitement, and competition, whereas those with low autonomous motivation focused on themes of wanting instrumental support and financial incentives for increasing engagement in physical activity. Implications for developing physical activity intervention programs for African American women that are targeted toward differences in autonomous motivation are discussed.
Abstract: Publication date: Available online 7 September 2019Source: Evaluation and Program PlanningAuthor(s): Clemens Striebing, Evanthia Kalpazidou Schmidt, Rachel Palmén Program designers at local project level usually design interventions under conditions of scarce resources and bounded rationality lacking sufficient information about the process and impact mechanisms of the complex social programs they design and implement. This paper proposes a model for pragmatic ex ante evaluation using an innovative conceptual framework for practitioners working in the field to improve gender equality in research and innovation.The presented and applied model is pertinent to the theory-oriented evaluation tradition using a logic frame and a theory of change approach adapted to the special requirements of ex-ante evaluation. The model is illustrated by the case study of a program for the promotion of women entrepreneurship in the STEM fields.It is concluded that in a setting with limited resources and information, ex-ante evaluations should aim at defining clear conditions of proportionality and need to take into account the extent to which contextual factors hinder or facilitate the implementation of a program, and the specific institutional and system mechanisms at play, in order to enable proactive risk management from the beginning.
Abstract: Publication date: Available online 6 September 2019Source: Evaluation and Program PlanningAuthor(s): Michael Francis Norwood, Ali Lakhani, Annick Maujean, Martin Downes, Simone Fullagar, Michelle McIntyre, Jason Byrne, Anna Stewart, Bonnie L. Barber, Elizabeth Kendall A framework for assessing photographs for the emotional and social health of young people (SHAPE) is described and tested, within the context of a rural program. Two independent raters assessed the photographs of participants. To assess inter-rater reliability, Cohen’ K and Kendall’s W were calculated. The two reviewers’ assessments of photographs were in agreement. The assessment of emotional/behavioural display showed 82% agreement. Agreement between reviewers’ judgements of proxemics (W = .866), interaction (W = .722), engagement (W = .932) and overall impression (W = .804) were all significant (p
Abstract: Publication date: Available online 3 September 2019Source: Evaluation and Program PlanningAuthor(s): Jody Brook, Becci Akin As part of the process of examining their theory of change (TOC) and reflecting on grant making activity, one Midwestern foundation employed a multipronged strategy to assess 209 community-based mental health grants across seven years of funding. This article details the evaluation approach, which comprised these areas of the TOC: grantees’ use of evidence-based interventions, cultural competency, quality improvement, community collaboration, and use of integrated care. Inductive analyses identified grantees’ use of innovative practices, trauma-informed care, and use of validated instruments. In an iterative exercise spanning multiple years, the foundation and researchers found ways to gather information about community capacity and impact while simultaneously minimizing grantee data collection/reporting burden. Findings revealed that community agencies were improving in their uptake of evidence-based interventions and use of validated instruments. Community grantee use of trauma informed care also improved over the study period. Grantee desire to collaborate and coordinate services within the community was also strong. The research also revealed that in multiple domains (such as technology and non-profit operating costs) grantees needed support in building agency capacity. These findings shed light on which aspects of the foundation’s TOC were being addressed through funding, and which areas of the community needed additional support.
Abstract: Publication date: Available online 22 August 2019Source: Evaluation and Program PlanningAuthor(s): Hickey Sophie, Couchman Katrina, Stapleton Helen, Roe Yvette, Kildea Sue
Abstract: Publication date: Available online 11 August 2019Source: Evaluation and Program PlanningAuthor(s): E. Mc Hugh, M. Starke Specific aimsThis study explored feasibility and fidelity of an education program for parents with intellectual disability. The Australian Parenting Young Children intervention was translated and adapted for support workers in the Swedish social services.MethodFidelity was examined using implementation diaries tracking frequency of implementation of program activities and teaching approaches ( (over 14 months, completed by 27 support workers from 15 municipalities). Eighteen of these support workers and their managers (n = 12) completed questionnaires examining feasibility using Roger’s concepts of program compatibility and complexity.FindingsResults showed that implementation of Parenting Young Children was feasible, with overall positive attitudes to Evidence Based Practice, good program compatibility, low perceived program complexity, and high satisfaction with implementation support. The fidelity measures suggest problems in how the program is used by some support workers: parents received less than the recommended number of sessions and activities such as observation of parents and follow up, were infrequently used by support workers.DiscussionThe findings raise questions about how program support and education should be provided. Difficulties in defining fidelity within the flexibility of the Parenting Young Children program are also discussed.
Hybrid journal (It can contain Open Access articles)
[3182 journals]
