Abstract: Publication date: Available online 2 March 2020Source: Evaluation and Program PlanningAuthor(s): Jesús Miguel Rodríguez-Mantilla, Angélica Martínez-Zarzuelo, Francisco José Fernández-Cruz
Abstract: Publication date: Available online 2 March 2020Source: Evaluation and Program PlanningAuthor(s): Arielle Weir, Simon Kitto, Jennifer Smith, Justin Presseau, Ian Colman, Simon Hatcher
Abstract: Publication date: Available online 27 February 2020Source: Evaluation and Program PlanningAuthor(s): Margaret Paek, Connor Radkey, Sally Honeycutt, M. Kathleen Glynn
Abstract: Publication date: June 2020Source: Evaluation and Program Planning, Volume 80Author(s): Hugo Henrique Roth Cardoso, Adriana Dantas Gonçalves, Gustavo Dambiski Gomes de Carvalho, Hélio Gomes de Carvalho
Abstract: Publication date: April 2020Source: Evaluation and Program Planning, Volume 79Author(s): Clemens Striebing, Evanthia Kalpazidou Schmidt, Rachel Palmén, Florian Holzinger, Beata Nagy
Abstract: Publication date: April 2020Source: Evaluation and Program Planning, Volume 79Author(s): Kristin M. Van De Griend, Deborah L. Billings, Edward A. Frongillo, DeAnne K. Hilfinger Messias, Amy H. Crockett, Sarah Covington-Kolb
Abstract: Publication date: Available online 25 February 2020Source: Evaluation and Program PlanningAuthor(s): Lisanne S. Mulderij, Fieke Wolters, Kirsten T. Verkooijen, Maria A. Koelen, Stef Groenewoud, Annemarie Wagemakers
Abstract: Publication date: Available online 25 February 2020Source: Evaluation and Program PlanningAuthor(s): Yue Liang, Mengbing Du, Xiangxiao Wang, Xiwei Xu
Abstract: Publication date: Available online 25 February 2020Source: Evaluation and Program PlanningAuthor(s): Tanis J. Walch, Richard R. Rosenkranz, Michaela A. Schenkelberg, Bronwyn S. Fees, David A. Dzewaltowski
Abstract: Publication date: Available online 19 February 2020Source: Evaluation and Program PlanningAuthor(s): Catarina Castro, Teresa Pereira, J.C. Sá, Gilberto SantosAbstractThe Hospital Pharmacy of a Local Health Unit (LHU) in greater Oporto plays a major role in the community. In addition to providing medicines to the entire Internal Unit, it has created a service called "Hospital Ambulatory Pharmacy", which allows continued hospital treatments at home, in a more comfortable and familiar environment. This is mostly due uderfunding which has hindered recruitment of health professionals as well a growing concern for the comfort of patients as they convalesce. The Hospital Ambulatory Pharmacy’s main problems are both recurring stock-outs and over-stocks; the fact that medicines’s delivery routes are not completed, and that medical staff lacks awareness of the techniques to overcome the logistic issues involved. As a methodology, a context analysis was carried out, KPI's were created to quantify the problems and raise health professionals’ awareness of what was happening. Solutions for storage were analysed and visits were made to a central hospital to analyse the solutions adopted to solve the same problems. The most appropriate inventory management policies were considered for the organization of existing and further resources. We chose to create an Advanced Warehouse with Level Replenishment. The levels were defined considering the demand of each medicine and divided into three different classes to use the best strategy for each type of demand. Then, and with the help of the LHU IT team, the inventory management module of the information system was parameterized, so stock replenishment could be done electronically. All these changes resulted in a 30% reduction in stock-out. At the beginning of the study, stock-outs would reach peaks of 6%. These were reduced to 4.5% at a point at which professionals were already more aware of the logistic problem, and finally to 1.5%, in the final phase, at a time when the proposal had already been implemented. Over-stock was reduced by 66%. Delivery routes were stipulated, and compliance was verified. A novel service developed to increase the well-being of patients monitored at home which it is intended to improve the service provided to the users and at the same time reduce operational costs.As a contribution to the knowledge, in addition to the case study presented to be a novelty in terms of scientific articles published because it is a service implemented in the distribution of medications to patients at home, mostly with oncologic diseases. Several methods from Lean Thinking were used, such as, Kanban System and Advanced Warehouse for inventory management levelling and optimization techniques, as well as, operational KPIs, with the purpose to reach out the improvement of program planning and evaluation. This approach resulted in some improvements, namely, the increase of the quality of the healthcare services for the population. And, in terms to reduce stock and excess stock-outs, to reduce inventory costs, setting the delivery routes, raising the entire medical team’s awareness of the problem and committing to optimization a reality.
Abstract: Publication date: Available online 19 February 2020Source: Evaluation and Program PlanningAuthor(s): S. Evans, A. Dadich, B. Stout, D. PlathAbstractIn realist evaluation, where researchers aim to make program theories explicit, they can encounter competing explanations as to how programs work. Managing explanatory tensions from different sources of evidence in multi-stakeholder projects can challenge external evaluators, especially when access to pertinent data, like client records, is mediated by program stakeholders. In this article, we consider two central questions: how can program stakeholder motives shape a realist evaluation project; and how might realist evaluators respond to stakeholders’ belief-motive explanations, including those about program effectiveness, based on factors such as supererogatory commitment or trying together in good faith' Drawing on our realist evaluation of a service reform initiative involving multiple agencies, we describe stakeholder motives at key phases, highlighting a need for tactics and skills that help to manage explanatory tensions. In conclusion, the relevance of stakeholders’ belief-motive explanations (‘we believe the program works’) in realist evaluation is clarified and discussed.
Abstract: Publication date: Available online 14 February 2020Source: Evaluation and Program PlanningAuthor(s): Joseph Adrien Emmanuel Demes, Nathan Nickerson, Lambert Farand, Victor Becerril Montekio, Pilar Torres, Jean Geto Dube, Jean Garcia Coq, Marie-Pascale Pomey, Francois Champagne, Ernst Robert JasminAbstractBackgroundQuality improvement in Healthcare is the new fashion actually. Rich countries and developing countries alike are trying to implement quality improvement initiatives to improve their performance and the quality of care. There is evidence in the scientific literature that the existence of a “champion” can play an important role in the successful implementation of quality improvement strategies. Most of the time, people get stuck at the implementation level: they know what to do but they fail to execute it in the organizational setting. That’s where a champion can be useful to facilitate the success of the change. There is a paucity of research on the link between the champion and the implementation of quality improvement programs. The aim of the study was to investigate the perceptions of stakeholders about the characteristics and qualities of the champion that could facilitate the successful implementation of quality improvement programs in health care settings in Haiti.MethodsTwenty semi-structured individual interviews and one small group discussion (n = 4) with providers, administrators, directors, and NGO representatives were conducted during a six months period. The total sample size was twenty-four (n = 24). The methods were informed by grounded theory and the data were analyzed using the constant comparative method and thematic content analysis approach.ResultsThe analysis resulted in eleven themes describing the characteristics and qualities of a successful champion: Communication and persuasion, proactivity, humility, horizontal and collective leadership style, sense of responsibility and accountability, go-ahead type, empathy, dedication and motivation, ability to inspire and motivate people, have a vision, and encourage learning.ConclusionsIt is important to support and encourage champions in the health care systems in developing countries to master and exhibit those qualities and characteristics in order to lead a team for the successful implementation of quality improvement initiatives in the health facilities. More research is needed to understand how to better articulate those qualities in the context of the Haitian health care system.
Abstract: Publication date: Available online 14 February 2020Source: Evaluation and Program PlanningAuthor(s): Mélissa Goulet, Isabelle Archambault, Michel Janosz, Jean Bélanger, Sandra L. ChristensonAbstractMany dropout prevention programs have been developed and validated in the past decades. Yet, little is known about the contextual factors influencing the implementation of these programs. Implementation processes, such as school principal leadership and governmental funding, have been identified for their influence on program implementation, but the mutual or dynamic influence of these processes is yet to be understood. This study examines the processes involved in the implementation of Check & Connect (C&C), a well-established targeted dropout prevention program validated in several countries that aims at promoting the development of a significant relationship between at-risk students and a mentor as a way to prevent school dropout.Semi-structured interviews were conducted with 12 mentors and coordinators directly involved in C&C implementation. Analyses were conducted in two subsequent steps: thematic analyses first helped identify implementation processes described by respondents, and then synthetic case studies allowed us to build the implementation stories of distinct sites. This last step was conducted using the Planned Change Framework.This framework helped us to understand the complex dynamics of implementation processes in each site, which were associated with previously identified program outcomes, beyond implementation fidelity. Implications for future implementations of evidence-based programs in the school setting are discussed.
Abstract: Publication date: Available online 12 February 2020Source: Evaluation and Program PlanningAuthor(s): Patrícia Moura e Sá, Carolina FernandesAbstractThe growing proportion of senior citizens and the importance of encouraging active well-ageing behaviours explain the increasing efforts carried out by governments to develop exercise programmes particularly targeted to this group. The complexity associated with the running of these programmes and the resources committed make their quality assessment essential. The aim of the current paper is to validate and refine the use of the EFQM model in this context. A programme of a medium-sized municipality of the Centre region of Portugal was used as a case study to evaluate the adequacy and relevance of the EFQM model and to identify the strengths and weaknesses of the current programme based on the data collected from service users, instructors, municipal leaders and programme partners. High levels of customer satisfaction and loyalty, together with the clarity of programme mission and strategy emerged as main areas of strength. On the other hand, insufficient attention to the measurement of people satisfaction, as well as the leaders’ failure of to show appreciation for the efforts developed by the programme instructors are key areas for improvement.
Abstract: Publication date: Available online 12 February 2020Source: Evaluation and Program PlanningAuthor(s): Jane E. Palmer, Samantha C. Winter, Sarah McMahonAbstractThe purpose of this study was to examine the final analytic sample of a longitudinal randomized control trial (RCT) evaluation of a sexual violence prevention program at a university after facing challenges with the implementation of a self-generated identification code. The matched and unmatched samples (e.g., all unique surveys across all time periods) included 10,135 surveys. Eighty-eight percent of these surveys were matched into the final longitudinal dataset. Findings suggest that students with certain characteristics were more likely to be matched over time (i.e., students who participated in student government, Latino/a students, and Asian students). In addition, students who did not comply with RCT protocol were less likely to be matched. Student history of victimization or perpetration of sexual violence was not associated with being matched over time. This study provides recommendations for preventing matching problems in longitudinal studies, a process for rectifying matching issues and a critique of studies that do not address issues of matching-related sample bias in their final analytic sample.
Abstract: Publication date: Available online 12 February 2020Source: Evaluation and Program PlanningAuthor(s): Christina Peterson, Gary SkolitsAbstractValue for Money (VfM) is an evaluative question about the merit, worth, and significance of resource use in social programs. Although VfM is a critical component of evidence-based programming, it is often overlooked or avoided by evaluators and decision-makers. A framework for evaluating VfM across the dimensions of economy, effectiveness, efficiency, and equity has emerged in response to limitations of traditional economic evaluation. This framework for assessing VfM integrates methods for engaging stakeholders in evaluative thinking to increase acceptance and utilization of evaluations that address questions of resource use. In this review, we synthesize literature on the VfM framework and position it within a broader theory of Utilization-Focused Evaluation (UFE). We then examine mechanisms through which the VfM framework may contribute to increased evaluation use. Finally, we outline avenues for future research on VfM evaluation.
Abstract: Publication date: Available online 8 February 2020Source: Evaluation and Program PlanningAuthor(s): Ea Høg UtoftAbstractThis article explores linkages between organisation-specific cultural narratives and gender-equality programme planning through the lens of the ‘historicity’ concept. The article argues that to fully understand problem definitions, programme design and organisational change processes related to gender equality, scholars and practitioners cannot focus one-sidedly on expected outcomes and effects. We need to factor in organisational narratives as well, because gender equality actors never arrive at their work as ‘tabulae rasae’. A community of actors always draws on shared dispositions that give sense, direction and shape to their anticipations of the future hereby guiding their actions in the present. Based on an ethnography of a multi-national engineering company, the article shows how cultural narratives may in different ways serve as support factors for gender equality programme planning and implementation, if they are actively but mindfully engaged. This mindfulness is important as positive cultural narratives may entail problematic gender dimensions. On the other hand, negative cultural narratives may entail important learning outcomes that may benefit future gender equality initiatives. The analysis further points to the centrality of strategic communication, leadership commitment and comprehensive evaluation in order to mobilise the potential of cultural narratives as support factors to gender equality work. Finally, this article offers to scholars and practitioners a rich example of how to employ cultural analysis in relation to gender equality activities, and demonstrates the value of the insights produced by this analysis for the case company and its gender equality programme.
Abstract: Publication date: Available online 7 February 2020Source: Evaluation and Program PlanningAuthor(s): Chih-Hao Yang, Kuen-Chang LeeAbstractCorporate fraud risk management strategy has increasingly become a sustainable business development goal. Recent reforms in forensic accounting technology for corporate fraud risk management globally have opened up new avenues for corporate governance and internal control mechanism implementation. This study thus presents an integrated methodology for forensic accounting implementation to improve the identification of the strategy map relationship between the Balanced Scorecard (BSC)-based perspective and criteria, by combining multiple-criteria decision making (MCDM) with the Decision Making Trial and Evaluation Laboratory (DEMATEL) and the Analytic Network Process (ANP) techniques. The results have implications for corporate decision-makers to effectively fulfil corporate governance quality assurance and anti-fraud through a forensic accounting strategy map illustration. From the evaluation and planning perspective, the in-depth analysis of strategy map is useful to obtain an interrelationship that takes as its starting point the practice professions of the decision maker to improve existing strategy alternatives and focus on the valuable strategy paths. In the evaluation planning application, a strategy map of forensic accounting presents the knowledge regarding key indicators’ priorities to achieve satisfactory strategy planning and to practice forensic accounting development linked to fraud risk management in Taiwan.
Abstract: Publication date: Available online 6 February 2020Source: Evaluation and Program PlanningAuthor(s): Hyunil Kim, Melissa Jonson-Reid, Patricia Kohl, Chien-jen Chiang, Brett Drake, Derek Brown, Tim McBride, Shenyang GuoAbstractRecurrence of child maltreatment is a significant concern causing substantial individual, family and societal cost. Variable-based approaches to identifying targets for intervention may not reflect the reality that families may experience multiple co-occurring risks. An alternative approach was tested using baseline data from the National Survey of Child and Adolescent Well-being (NSCAW) I and II to develop Latent Class Analysis models of family risk classes using variables derived from prior studies of re-reporting. The samples were collected approximately 10 years apart offering a chance to test how the approach might be impacted by demographic or policy shifts. The association between baseline classes and later re-reports was tested using both samples. A two-class model of high versus low presence of baseline risk resulted that was strongly associated with later likelihood of re-report and results were relatively stable across the two studies. Person-centered approaches may hold promise in the early identification of families that require a more comprehensive array of supports to prevent re-reports of maltreatment.
Abstract: Publication date: Available online 4 February 2020Source: Evaluation and Program PlanningAuthor(s): John Diaz, Anil Kumar Chaudhary, K.S.U. Jayaratne, Elsie AssanAbstractThe overlap of competencies between general program evaluation and specific contexts or content will always be reality because evaluators may need unique competencies to answer evaluation questions for particular contexts or content areas. Limited research exists that explores the essential competencies required by professionals who use evaluation as one part of their job portfolio, which leaves unanswered questions regarding the applicability of current evaluator competency models in such settings. We used a modified three-round Delphi technique to identify evaluator competencies for non-formal educators in Cooperative Extension (CE). Our panelists identified 36 competencies in the non-formal educational programming context for CE educators that they considered important to be included in evaluation capacity building efforts. We categorized our 36 identified competencies from the Delphi study into the five competency domains proposed by the American Evaluation Association. Our findings provide information to help guide professional development among non-formal educators related to program evaluation.
Abstract: Publication date: Available online 1 February 2020Source: Evaluation and Program PlanningAuthor(s): Megan S. Patterson, Tyler Prochnow, Ryan G. Richardson, Kevin P. JacksonAbstractObjectiveTo conduct a system-wide assessment using social network analysis (SNA) to examine how 14 important issues (e.g., consent; racism) are addressed through education, training, and programming at a university.MethodsEvaluators conducted interviews with campus departments responsible for educating/training on the 14 issues. Interviews revealed which programs (n = 52) were offered that addressed the 14 issues, and data on audience characteristics, date of delivery, and which issues were covered in each program were collected. SNA was used to calculate degree and create visualization graphs illustrating patterns of content-coverage across all 52 programs.ResultsThe average degree was 19.38 (SD = 9.70), meaning programs overlapped in topic area with nearly 20 other programs, on average. Most programs (n = 36; 69.2%) were attended by audiences of 500 people or less. “Diversity and inclusion” represented the topic area with the most programs (n = 23), whereas “suicide” and “bullying/hazing” had the least number of programs (n = 3). Degree was negatively correlated with attendance numbers (r=-.310, p
Abstract: Publication date: Available online 27 January 2020Source: Evaluation and Program PlanningAuthor(s): Cheryl N. Poth, Michelle Searle, Alexandra M. Aquilina, Jenny Ge, Alexa Elder
Abstract: Publication date: Available online 27 January 2020Source: Evaluation and Program PlanningAuthor(s): Moonjung Yim, Michelle Fellows, Chris CowardAbstractThe article presents a mixed-methods evaluation of regional libraries in Namibia, which incorporates three perspectives: the patron perspective (library users), the library perspective (library staff, management, and related officials), and the external perspective (including evaluators and monitoring data). Seven data collection methods were used: patron surveys, patron panel studies, focus group discussions, key informant and staff interviews, secondary data analysis, media analysis, and observations. The goal of the evaluation was to assess library performance for both formative and summative purposes by addressing evaluation questions on areas such as library services, use, and operations. Building upon the literature review of how mixed-methods approaches can contribute to library evaluation, the aim of this article is to show how a mixed-methods evaluation can be designed to examine multi-faceted library performance and to illustrate how the evaluation design allows information complementarity and can be utilized to present diverse viewpoints of the above three perspectives. The evaluation design, analysis process, and lessons learned from this study may be useful to evaluators engaged in evaluation of public services or programs (including public libraries) that examine multiple aspects of service performance and involve a variety of stakeholders.
Abstract: Publication date: Available online 26 January 2020Source: Evaluation and Program PlanningAuthor(s): John M. LaVelle, Chris Lovato, Clayton StephensonAbstractPedagogy and andragogy refer to the philosophical and empirical assumptions that instructors make about what and how people learn. These assumptions guide much of the subsequent instructional design process, including learning objectives, learning experiences, assessment techniques, and more. Most of the existing literature in evaluator education is descriptive in nature, and there has been limited attention to research on pedagogy specific to evaluation or identification of effective practices for teaching evaluation. As a result, we know very little about the theoretical or practical pedagogy of evaluation. As a first step in understanding this pedagogy, this article first synthesizes relevant concepts, principles of learning, and models of instructional design principles of learning developed in the field of education. The authors then offer case examples of how these ideas, principles, models, and pedagogies apply to five evaluation-specific courses.
Abstract: Publication date: Available online 25 January 2020Source: Evaluation and Program PlanningAuthor(s): Sarah MasonAbstractEvaluation approaches should be appropriate for their contexts. Scholars and practitioners alike have widely acknowledged this view for at least four decades, and the ability to respond to context is clearly established as a core evaluator competency. Outside of evaluation, this knowledge of context, known as situation awareness, is seen as a critical feature of good decision making and as a factor that distinguishes experts from novices across a wide range of domains. Yet there are few opportunities for evaluators to explicitly build their skills in situation awareness. This study addressed that gap by examining the potential for one online training program, informed by research on deliberate practice, to accelerate evaluators’ progress towards expertise in situation awareness. Built to align with evidence on developing situation awareness skills, the EvalPractice portal combined (1) an extensive ‘case bank’ of real-world evaluation scenarios, (2) repetitive practice that allowed novice evaluators to practice interpreting these evaluation scenarios, and (3) immediate feedback on the accuracy of these efforts based on events from the real-life version of the scenario. Findings from a small EvalPractice pilot suggest that it may be possible to improve foundational situation awareness skills using deliberate practice, but that further research is required to understand strategies for building higher-level situation awareness skills in evaluation.
Abstract: Publication date: Available online 22 January 2020Source: Evaluation and Program PlanningAuthor(s): Anna Newton-Levinson, Megan Higdon, Jessica Sales, Laurie Gaydos, Roger RochatAbstractThe need for conducting evaluations which reflect of the influence of context on complex programs is increasingly recognized in the field of evaluation. Better data visualization techniques for connecting context with program evaluation data are needed. We share our experience developing a mixed methods timeline to visualize complexity and context with evaluation data. Mixed methods timelines provide a meaningful way to show change over time in both a visually stimulating and accessible format for evaluation audiences. This paper provides an innovative example of using mixed methods timelines to integrate evaluation data with key program activities and milestones, while also showing internal and external contextual influences in one cohesive visual. We present methods and best practices for collecting contextual data and for incorporating a variety of data sources into such a visual. We discuss several strategies to collect and organize context related data including: qualitative interviews, program materials, narrative reports, and member checking with stakeholders and staff. Gathering multiple perspectives is essential to better capture the multi-layered elements of program activities and context.
Abstract: Publication date: Available online 21 January 2020Source: Evaluation and Program PlanningAuthor(s): Randi M. Williams, Jing Zhang, Nathaniel Woodard, Jimmie Slade, Sherie Lou Zara Santos, Cheryl L. KnottAbstractInstitutionalization of health promotion interventions occurs when the organization makes changes to support the program as a component of its routine operations. To date there has not been a way to systematically measure institutionalization of health promotion interventions outside of healthcare settings. The purpose of the present study was to develop and evaluate the initial psychometric properties of an instrument to assess institutionalization (i.e., integration) of health activities into faith-based organizations (i.e., churches). This process was informed by previous institutionalization models led by a team of experts and a community-based advisory panel. We recruited African American church leaders (N = 91) to complete a 22-item instrument. An exploratory factor analysis revealed four factors: 1) Organizational Structures (e.g., existing health ministry, health team), 2) Organizational Processes (e.g., records on health activities; instituted health policy), 3) Organizational Resources (e.g., health promotion budget; space for health activities), and 4) Organizational Communication (e.g., health content in church bulletins, discussion of health within sermons) that explained 62.3% of the variance. The measure, the Faith-Based Organization Health Integration Inventory (FBO-HII), had excellent internal consistency reliability (α = .89) including the subscales (α = .90, .82, .81, and .87). This measure has promising initial psychometric properties for assessing institutionalization of health promotion interventions in faith-based settings.
Abstract: Publication date: April 2020Source: Evaluation and Program Planning, Volume 79Author(s): Jay Wade, Leanne KallemeynAbstractEvaluation capacity building (ECB) is a practice that can help organizations conduct and use evaluations; however, there is little research on the sustainable impact of ECB interventions. This study provides an empirical inquiry into how ECB develops sustained evaluation practice. Interviews were conducted with 15 organizational leaders from non-profits, higher education institutions, and foundations that “bought in” to ECB and were at least six months removed from an ECB contract. The result of this work highlights how sustained evaluation practice developed over time and what these practices looked like in real-world settings. A developmental, iterative cycle for how ECB led organizations to sustain evaluation practice emerged around key components to sustainability. First, leadership supported ECB work and resources were dedicated to evaluation. Staff began to conduct and use evaluation, which led to understanding the benefits of evaluation, and promoted value and buy-in to evaluation among staff. Common barriers and emerging sustainability supports not previously identified by ECB literature—the “personal” factor and ongoing ECB practitioner contact—are described. Practical tips for ECB practitioners to promote sustainability are also detailed.
Abstract: Publication date: April 2020Source: Evaluation and Program Planning, Volume 79Author(s): Ruby A. Natale, Ellen Kolomeyer, Ana Robleto, Zafreen Jaffery, Rachel SpectorAbstractSocial-emotional issues in preschoolers continue to be an area of concern across the nation. Models to determine effective implementation practices are needed. The Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework is one model that can be used to evaluate program implementation in preschool settings. The Jump Start program provided short-term intervention to children in low-income preschools from multiethnic backgrounds. The goal was to promote prosocial behaviors while minimizing problem behaviors. Various evidenced-based practices (i.e., I Can Problem Solve, play therapy, Pyramid Model) were utilized. 305 children from 73 childcare centers participated in the program. The majority of participants were from ethnic minority backgrounds and resided in high-poverty areas of the county. The RE-AIM framework was utilized to determine program outcomes. Results showed successes on each level of RE-AIM with an at-risk population. The Jump Start program significantly increased positive behaviors and decreased challenging behaviors in preschool-aged children. Results indicated medium to large effect sizes. One year following program participation, the majority of children who participated in the program were at decreased risk of special education services and expulsion.
Abstract: Publication date: April 2020Source: Evaluation and Program Planning, Volume 79Author(s): Erin Stern, Ingrid van der Heijden, Kristin DunkleAbstractWomen with disabilities are more vulnerable to violence, including intimate partner violence (IPV), yet the majority of emerging IPV prevention programs fail to explicitly consider the needs of participants with disabilities. Women and men living with disabilities engaged with IPV prevention programs in four countries were interviewed to explore how disability shaped their experiences of gender, violence, IPV, and whether the programs met their disability related needs. In-depth interviews were conducted with 16 women and 15 men living with disabilities in Ghana, Rwanda, Tajikistan and South Africa. The data were analysed thematically and compared across the settings. Participants described experiencing disability-related stigma, discrimination, exclusion, and for women, increased vulnerability to IPV. Barriers to full participation in programs included limited accessibility, and lack of disability-specific materials, recruitment or outreach. Enablers of inclusion included recruitment and monitoring strategies aimed at people with disabilities, partnering with a local disabled people’s organization, training staff in disability inclusion, and raising awareness of disability rights. The data encouragingly suggests that inclusion of women and men with disabilities in IPV prevention programs designed for the general population has beneficial outcomes. Inclusion can prevent violence, promote their wellbeing, support economic empowerment, and challenge disability-related stigma and discrimination.
Abstract: Publication date: April 2020Source: Evaluation and Program Planning, Volume 79Author(s): Jill R. Reed, Danae Dinkel, Matthew R. Bice, Paul Estabrooks, Bunny Pozehl, Kate HeelanAbstractAddressing the lack of physical activity (PA) in rural adults is vital because of the role it plays in the risk for many chronic diseases. The purpose of the study was to explore the feasibility of conducting a 12-week intervention utilizing a Fitbit to increase PA behavior in inactive rural adults recruited from a primary care clinic. Semi-structured interviews were conducted with study participants (n=10) and a focus group was held with nurses (n=7) from the primary care clinic. Data were analyzed using the process of immersion/crystallization. Participants and nurses both felt the program had a positive impact on PA and that the program was needed in the community. Study participants most often reported their favorite part of the study was being able to track their activity as well as the goal setting. However, study participants reported barriers to continued participation in PA related to a lack of time and ability to be active in cold weather. Further, nursing staff reported barriers in regard to a lack of resources (staffing and money) These barriers need to be examined further and addressed in order to implement a sustainable PA program that can be maintained through a primary care clinic.
Abstract: Publication date: Available online 3 January 2020Source: Evaluation and Program PlanningAuthor(s): Christian Dagenais, Didier Dupont, Frédéric Nault-Brière, Diego Mena, Gabrielle Yale-Soulière, Esther Mc Sween-Cadieux
Abstract: Publication date: Available online 20 December 2019Source: Evaluation and Program PlanningAuthor(s): Parthenia Dinora, Andrew Schoeneman, Molly Dellinger-Wray, Elizabeth Cramer, Jack Brandt, Allison D’Aguilar
Abstract: Publication date: Available online 20 December 2019Source: Evaluation and Program PlanningAuthor(s): Susanne Bührer, Rainer FrietschAbstractThis article examines whether two of the major German flagship programmes to increase the participation of female researchers in the German science system, the "Women Professorship Programme" and the "Pact for Research and Innovation", have actually increased the number of women, especially in leadership positions. In a second step, we analyse whether such an assumed increase influences the publication patterns of authors with German affiliation. This article is based on literature and desk research as well as bibliometric analysis using Scopus. The most important result is that the number of women in research has indeed increased significantly in recent years and that, accordingly, more women are the (co)authors of scientific publications. In particular, it can be seen that quality indicators such as citations and excellence rates are high for female authors. This enables us to show that more women in the science system not only bring about a "gain in justice", but also a concrete scientific benefit.
Abstract: Publication date: Available online 9 December 2019Source: Evaluation and Program PlanningAuthor(s): Anil Kumar Chaudhary, John Diaz, K.S.U. Jayaratne, Elsie AssanAbstractPolicymakers’ demand for increased accountability has compelled organizations to pay more attention to internal evaluation capacity building (ECB). The existing literature about ECB has focused on capacity building experiences and organizational research, with limited attention on challenges that internal evaluation specialists face in building organizational evaluative capacity. To address this knowledge gap, we conducted a Delphi study with evaluation specialists in the United States’ Cooperative Extension Service and developed a consensus on the most pervasive ECB challenges as well as the most useful strategies for overcoming ECB challenges. Challenges identified in this study include limited time and resources, limited understanding of the value of evaluation, evaluation considered as an afterthought, and limited support and buy-in from administrators. Alternatively, strategies found in the study include a shift in an organizational culture where evaluation is appreciated, buy-in and support from administration, clarifying the importance of quality than quantity of evaluations, and a strategic approach to ECB. The challenges identified in this study have persisted for decades, meaning administrators must understand the persistence of these issues and make an earnest investment (financial and human resource) to make noticeable progress. The Delphi approach can be used more often to prioritize ECB efforts.
Abstract: Publication date: Available online 27 November 2019Source: Evaluation and Program PlanningAuthor(s): Jean Marie Buregeya, Christine Loignon, Astrid BrousselleAbstractThe health impact assessment (HIA) is a tool used to estimate the potential impact on health of non-health-related proposals prior to implementation. While it is increasingly used in Quebec, Canada, studies have not analyzed its medium-term impacts and potential long-term impacts. We conducted a contribution analysis using in-depth interviews with key stakeholders, as well as documents, observation and images related to HIA in order to analyze its impacts on the revitalization of road infrastructure, parks and green spaces, and residential housing. Our analysis not only reflects on the decision-making process through the adoption and implementation of HIA recommendations, but also on the link between actions implemented in the field and health outcomes.RésuméL’évaluation d’impact sur la santé (ÉIS) estime l’impact potentiel sur la santé des actions hors du champ de la santé avant leur implantation. Bien qu’elle soit de plus en plus utilisée au Canada, en particulier au Québec, les études n’ont pas analysé ses impacts intermédiaires et ses effets potentiels à long-terme. Nous avons appliqué l’analyse de contribution à l’aide d’entretiens approfondis avec les principales parties prenantes, les documents, l’observation et les images liées à l’ÉIS pour analyser ses effets sur la revitalisation des infrastructures routières, des parcs et espaces verts et des logements résidentiels. L’analyse des effets de l’évaluation d’impact sur la santé porte davantage sur le processus décisionnel en rapport à l’adoption et l’implantation des recommandations issues de l’ÉIS, mais aussi sur le lien entre les actions mises en œuvre sur le terrain et les résultats en matière de santé.
Abstract: Publication date: Available online 22 November 2019Source: Evaluation and Program PlanningAuthor(s): Catherine Hudon, Maud-Christine Chouinard, Astrid Brousselle, Mathieu Bisson, Alya DanishAbstractCase management programs for frequent users of healthcare services are complex interventions which implementation and application are challenging to evaluate. The aim of this article was to conduct a logic analysis to evaluate a case management program for frequent users of healthcare services. The study proceeded in three phases: 1) establishing causal links between the program’s components by the construction of a logic model, 2) developing an integrated framework from a realistic synthesis, and 3) making a new reading of the case management program in regard of the integrated framework. The study demonstrated, on one hand, strengths and weaknesses of the actual case management program, and, on the other hand, how logic analysis can create a constructive dialogue between theory and practice. The evaluative process with decision-makers, clinicians and patients has helped to make connexions between theory, practice, experience and services organization.
Abstract: Publication date: Available online 12 November 2019Source: Evaluation and Program PlanningAuthor(s): Evanthia Kalpazidou Schmidt, Ebbe Krogh GraversenAbstractIn this article, we discuss the development of a conceptual evaluation framework to design and assess gender equality interventions and their effects in research and innovation. The conceptual framework presented herewith embraces the complexity, gender-sensitive and theory-based evaluation approaches ensuring that design and evaluation of gender equality interventions consider the complex systems that constitute the context in which the interventions operate. The evaluation framework offers a non-linear concept, where the notion of contribution - not attribution - to achieve impact is central to the integration of team, organizational and system factors in policy design and evaluation. The paper opens the “black box” to address the question of how and why a policy intervention works and in which context and discusses a systematic process on how to approach the interwoven linkages between input, implementation and effects in gender equality interventions in research and innovation, accounting for context sensitivity and methodological pluralism. The evaluation framework may serve as reference for researchers, evaluators, policymakers and other stakeholders in designing and assessing gender equality interventions, and in further developing their evidence, and theoretical and methodological base.
Abstract: Publication date: Available online 12 November 2019Source: Evaluation and Program PlanningAuthor(s): Rachel Palmén, Lidia Arroyo, Jörg Müller, Sybille Reidl, Maria Caprile, Maximillian UngerAbstractGender equality and gender mainstreaming in research is one of the six European Research Area (ERA) priorities. Integrating the gender dimension in research content and teaching is one of its three objectives. It is arguably the objective where least progress has been made. In this article we contribute to the evidence base by applying the EFFORTI evaluation framework to three empirical case study interventions that aim to integrate the gender dimension in tertiary education and research content. Comparison is based on an evaluation of the design of the intervention, those factors that have enabled/ hindered its implementation as well as an assessment of outcomes and impacts. The findings of the case studies highlight the importance of design, specifically regarding resources, legal status and the definition and operationalisation of the gender concept. Implementation hinges on top-level institutional commitment and mainstreaming gender studies with support of a central unit and crucially gender competence. A lack of recognition and status of gender studies and innovations was seen to hamper implementation. Outcomes and impacts included an increased awareness and interest in gender, increased gender competence, a push towards gender equality regarding representation and organisational change as well as an improved accreditation process and more and better research.
Abstract: Publication date: Available online 11 November 2019Source: Evaluation and Program PlanningAuthor(s): Susanne Bührer, Rainer FrietschAbstractThis article examines whether two of the major German flagship programmes to increase the participation of female researchers in the German science system, the "Women Professorship Programme" and the "Pact for Research and Innovation", have actually increased the number of women, especially in leadership positions. In a second step, we analyse whether such an assumed increase influences the publication patterns of authors with German affiliation. This article is based on literature and desk research as well as bibliometric analysis using Scopus. The most important result is that the number of women in research has indeed increased significantly in recent years and that, accordingly, more women are the (co)authors of scientific publications. In particular, it can be seen that quality indicators such as citations and excellence rates are high for female authors. This enables us to show that more women in the science system not only bring about a "gain in justice", but also a concrete scientific benefit.
Abstract: Publication date: Available online 7 November 2019Source: Evaluation and Program PlanningAuthor(s): Joshua Brisson, Igor Pekelny, Michael UngarAbstractThis article reflects on common challenges and lessons learned during the evaluation of gang prevention programs based on case studies of three federally funded Canadian programs. Elements of evaluation design, implementation, data analysis and reporting of results are discussed. More specifically, the article highlights issues that occur when evaluating community projects focused on preventing extreme risks for violence and the complexity of working in potentially dangerous and/or unstable work environments. Topics covered include the problem with quasi-experimental designs, model fidelity adherence, program documentation, client recruitment and retention, and data collection. Recommendations are provided to improve evaluations of youth gang prevention programs and similar community-based interventions that focus on the prevention of youth violence.
Abstract: Publication date: Available online 6 November 2019Source: Evaluation and Program PlanningAuthor(s): M.E. Menconi, S. Tasso, M. Santinelli, D. GrohmannAbstractUrban parks are often designed and renewed using standardized models that fail to incorporate the different functions and services required by the local communities served. Furthermore, when the citizens are involved in co-design, the participation processes are seldom fully inclusive, and the results are not representative of the entire local community. The paper starts from these considerations for developing an innovative participatory design method to renew urban parks. This method simultaneously uses multiple and alternative participation tools for adapting to the different technical skills of the citizens and to their inclinations. Furthermore, it entails the analysis and interpretation of the expectations of the local community in light of the urban green network. The method uses the conceptual framework of the SWOT analysis and the Trochim's concept mapping for developing a card game playable face to face or online. The results of the card game are processed using the statistical analysis to identify the main design themes (the clusters), and to split the participants’ suggestions into different design alternatives. Successively, the participants vote these alternatives for selecting the renewal design respondent to the local community's needs.
Abstract: Publication date: Available online 11 August 2019Source: Evaluation and Program PlanningAuthor(s): E. Mc Hugh, M. StarkeAbstractSpecific aimsThis study explored feasibility and fidelity of an education program for parents with intellectual disability. The Australian Parenting Young Children intervention was translated and adapted for support workers in the Swedish social services.MethodFidelity was examined using implementation diaries tracking frequency of implementation of program activities and teaching approaches ( (over 14 months, completed by 27 support workers from 15 municipalities). Eighteen of these support workers and their managers (n = 12) completed questionnaires examining feasibility using Roger’s concepts of program compatibility and complexity.FindingsResults showed that implementation of Parenting Young Children was feasible, with overall positive attitudes to Evidence Based Practice, good program compatibility, low perceived program complexity, and high satisfaction with implementation support. The fidelity measures suggest problems in how the program is used by some support workers: parents received less than the recommended number of sessions and activities such as observation of parents and follow up, were infrequently used by support workers.DiscussionThe findings raise questions about how program support and education should be provided. Difficulties in defining fidelity within the flexibility of the Parenting Young Children program are also discussed.
Hybrid journal (It can contain Open Access articles)
[3203 journals]
