Abstract: André, Nathalie et al. -- Keywords: Validité, Validation psychométrique, Modèles factoriels, Modèles à facettes de Guttman Validity, Psychometric test validation. -- Abstract : Validité, validation d'un test, propriétés ou qualités psychométriques, voilà trois expressions qui désignent les opérations ou procédures permettant de juger de la pertinence d'un instrument de mesure psychométrique, de son élaboration et son application. L'expression ``validité psychométrique'' a dès les débuts reçu diverses interprétations et elle continue aujourd'hui d'interpeller les théoriciens. La validité d'un test réside-t-elle dans la mesure qu'il produit ou dans l'action qui en découle ou bien réfère-t-elle à l'interprétation qu'on lui attache' La validité repose-t-elle strictement sur le résultat d'une procédure de validation' Un test ``démontré valide'' est-il valide en lui-même ou sa ``validité'' est-elle relative à la démonstration qu'on en a faite ou à son usage prévu' Existe-t-il différentes espèces de validité ou représente-t-elle un concept unifié, molaire' Une définition claire et consensuelle du concept de ``validité psychométrique'' n'a pas encore vu le jour. C'est dans une perspective à la fois pragmatique et conceptuellement rigoureuse que nous préconisons d'abolir l'expression ``validité psychométrique'', que cette expression ne correspond pas \emph {per se} à un concept, mais qu'elle est tout simplement un jugement, tout comme l'est au tribunal le prononcé du juge en considération des preuves qu'on lui soumet, de l'intimé concerné et des circonstances attenantes au procès. La validité est un jugement à l'effet que la mesure, le test, l'échelle psychométrique répond adéquatement et fidèlement à l'usage qu'on lui destine. L'essai explore ensuite le riche arsenal disponible au psychométricien et à l'utilisateur pour leur permettre de ``juger'' à bon escient de la validité situationnelle du test produit. \\ Validity, test validation, psychometric properties or qualities: here are three expressions that designate the operations and procedures that make it possible to judge the relevance of a psychometric measurement instrument, its development and its use. The expression ``psychometric validity'' has historically inherited various interpretations from the outset and it continues to challenge theorists today. Does the validity of a test lie in the measure it produces or in the action that is derived from it, or does it refer to the interpretation intended for it' Is validity based strictly on the result of a validation procedure' Is a "demonstrated valid" test valid in itself or is its "validity" relative to the demonstration or intended use' Are there different species of validity or does it represent a unified, molar concept' A clear and consensual definition of the concept of "psychometric validity" has not yet been found. It is from a pragmatic yet conceptually rigorous perspective that this essay argues for the abolition of the term "psychometric validity", that the term is not a concept \emph {per se}, but is simply a judgment, just as a judge's pronouncement in court is a judgment in light of the evidence before him or her, the respondent, and the circumstances surrounding the trial. Validity is a judgment that the measure, test, or psychometric scale responds adequately and reliably to its intended use. The essay then explores the well-stocked arsenal available to the psychometrician and his client to enable them to ``judge'' the situational validity of the test produced.
Abstract: Effatpanah, Farshad et al. -- Keywords: Reduced redundancy, speeded cloze-elide test, scoring methods, item response theory, Rash partial credit model. -- Abstract : Cloze-elide tests are overall measures of both first (L1) and second language (L2) reading comprehension and communicative skills. Research has shown that a time constraint is an effective method to understand individual differences and increase the reliability and validity of tests. The purpose of this study is to investigate the psychometric quality of a speeded cloze-elide test using a ploytomous Rasch model, called partial credit model (PCM), by inspecting the fit of four different scoring techniques. To this end, responses of 150 English as a foreign language (EFL) students to a speeded cloze-elide test was analyzed. The comparison of different scoring techniques revealed that scoring based on wrong scores can better explain variability in the data. The results of PCM indicated that the assumptions of unidimensionality holds for the speeded cloze-elide test. However, the results of partial credit analysis of data structure revealed that a number of categories do not increase with category values. Finally, suggestions for further research, to better take advantage of the flexibilities of item response theory and Rasch models for explaining count data, will be presented.
Abstract: Sambaraju, Prasanth -- Keywords: Power of chi-square test, cumulative distribution function, noncentral. -- Abstract : Noncentral distributions are obtained by transformation of their respective central distributions, and are identified by a noncentrality parameter. The noncentrality parameter measures the degree to which mean of test statistics departs, when the null hypothesis is false. Central distributions are used to describe test statistics, when the null hypothesis is true. Noncentral distributions are used to calculate statistical power of a test in situations when the null hypothesis is false. The paper presents Visual Basic for Application code in Microsoft Excel to compute the cumulative distribution function for noncentral chi-square distributions. The results obtained were found to be comparable with the reported values.
Abstract: Clement, Leah Mary et al. -- Keywords: moderation, mediation, moderated mediation, tutorial, data analysis. -- Abstract : Interest in moderation and mediation models have gained momentum since the 1980s and have become widespread in numerous fields of research including clinical, social, and health psychology in addition to behavioral, educational, and organizational research. There are resources available to help the user understand an analysis of a moderated mediation using the PROCESS macro and its resultant output, however, many are in video format (e.g., YouTube) or lack detailed instructions based on real world examples. To our knowledge, there are no resources that provide a thorough yet accessible step-by-step explanation of the procedure involved in using PROCESS v4.1 to analyze and interpret a moderated mediation model using real data in SPSS v28. The aim of this guide is to address this knowledge gap. An overview of mediation, moderation, and moderated mediation models is presented followed by instructions for verifying that assumptions are respected. Finally, a procedure to analyze data using PROCESS v4.1 is presented along with an interpretation of the resultant output.
Abstract: Bhaskar, Adhin et al. -- Keywords: count data; Poisson; negative binomial; zero inflation; hurdle regression. -- Abstract : Presence of excess zeros and the distributions are major concern in modeling count data. Zero inflated and hurdle models are regression techniques which can handle zero inflated count data. This study compares various count regression models for survey data observed with excess zeros. The data for the study is obtained from a survey conducted to assess the harms attributable to drinkers among children. Poisson, negative binomial and their zero inflated and hurdle versions were compared by fitting them to two count response variables, number of physical and number of psychological harms. The models were compared using fit indices, residual analysis and predicted values. The robustness of the models were also compared using simulated data sets. Results indicated that the Poisson regression was less robust to deviations from the distributional assumptions. The negative binomial regression and hurdle regression model were found to be suitable to model the number of physical and number of psychological harms respectively. The results showed that excess zeros in count data does not imply zero inflation. The zero inflated or hurdle models are suitable for zero inflated data. The selection between the zero inflated and hurdle models should be based on the assumed cause of zeros.
Abstract: Trudel, Marissa -- Keywords: Semantic analyses; book content; statistics books. -- Abstract : Le logiciel Sémato développé par Pierre Plante à l'Université du Québec à Montréal permet de compléter des analyses sémantiques avec de très larges données textuelles. Cet article a comme but d'être un guide pour faciliter l'utilisation du logiciel par de futurs chercheurs. Des instructions étape par étape sont fournies pour (1) la préparation du texte avant l'analyse; (2) l'ouverture d'un nouveau projet; (3) l'identification, la modification et l'élimination des thèmes; et (4) certaines analyses. Ces explications sont supportées par des figures d'une analyse précédente qui illustrent quelles étapes suivre sur le logiciel.
Abstract: Sharpe, Donald et al. -- Keywords: SPSS, ANOVA, Multilevel Models, Hierarchical Models, Mixed Models. -- Abstract : The treatment-control pre-post-follow-up (TCPPF) design is a popular means to demonstrate that a treatment group is superior to a control group over time. The TCPPF design can be analyzed using traditional methods (e.~g., between-within ANOVA) or with modern multilevel (also known as mixed or hierarchical) modeling. In spite of TCPPF’s widespread popularity, there is sparse and confusing guidance for applied researchers on how to analyze data from TCPPF designs using SPSS, one of the most popular software packages for data analysis. We present an introductory tutorial on methods for analyzing TCPPF data. Advantages, disadvantages, and cautions related to applying these approaches are discussed.
Abstract: Parisien, Michel -- Keywords: MMPI/MMPI-2, RC scales/MMPI-2-RF, empirical versus factor-theoretic validity, MMPI-3. -- Abstract : The conceptual-factorial scales of the MMPI-2-RC were constructed in divorce from the empirical categorizing strategy used for the original MMPI and its updated version, the MMPI-2. They were then integrated as a modern asset into a brand new instrument also based on factor analysis, the MMPI-2-RF. The latter was first introduced as a parallel version rather than a substitute for the MMPI/MMPI-2. However, workshops and webinars were multiplied, extolling the superiority of the new test. These subterfuges were then abandoned with the announcement of an MMPI-3, in the fall of 2020. This article, both historical and critical, takes stock of the MMPI-2, comments on the incongruous appearance of the "restructured scales" RC, summarily describes the restructured form MMPI-2-RF, and denounces the dreaded but predicted discarding of the MMPI-2 in favor of the MMPI-2-RF/MMPI-3, heretical avatars of MMPI that have recently surfaced in the market.
Abstract: Alter, Udi et al. -- Keywords: Equivalence testing, negligible effect, linear regression, lack of association. -- Abstract : Psychological research is rife with inappropriately concluding “no effect” between predictors and outcome in regression models following statistically nonsignificant results. However, this approach is methodologically flawed because failing to reject the null hypothesis using traditional, difference-based tests does not mean the null is true. Using this approach leads to high rates of incorrect conclusions that flood psychological literature. This paper introduces a novel, methodologically sound alternative. In this paper, we demonstrate how an equivalence testing approach can be applied to multiple regression (which we refer to here as “negligible effect testing”) to evaluate whether a predictor (measured in standardized or unstandardized units) has a negligible association with the outcome. In the first part of the paper, we evaluate the performance of two equivalence-based techniques and compare them to the traditional, difference-based test via a Monte Carlo simulation study. In the second part of the paper, we use examples from the literature to illustrate how researchers can implement the recommended negligible effect testing methods in their own work using open-access and user-friendly tools (negligible R package and Shiny app). Finally, we discuss how to report and interpret results from negligible effect testing and provide practical recommendations for best research practices based on the simulation results. All materials, including R code, results, and additional resources, are publicly available on the Open Science Framework (OSF): \href {https://osf.io/w96xe/}{osf.io/w96xe/}.
Abstract: Fox, Elizabeth L. et al. -- Keywords: Multi-tasking, cognitive workload, redundant signal processing. -- Abstract : In high demand contexts, uni- or multi-modal signals are used to convey redundant information and improve performance. This is especially the case with improving the detection of discrete peripheral signals. However, how one processes peripheral signals may change depending on the greater environmental context. The underlying cognitive processing of signals is important to determine how they may influence the degree to which each signal enhances, as opposed to slows down, detection. Until now, it was unclear if i) the introduction of, or increased difficulty of, a second task changes how people combine peripheral signals (that is, in a parallel, serial, or coactive fashion) and ii) if processing efficiency depends on the salience of the peripheral signals or the presence/difficulty of a centrally located and continuous tracking task. This manuscript describes an application of Systems Factorial Technology to investigate the cognitive processing mechanisms of redundant signals in the context of a multiple object tracking (MOT) task. The MOT task load (track 0, 1, or 4 dots) and the salience of peripheral signals (bright, dim) were manipulated. The data indicate peoples' processing of peripheral signals changed depending on the MOT task load. Under a high MOT task load, most people processed redundant signals in a parallel fashion. Alternatively, nearly half of people processed the signals in a serial fashion when asked to simultaneously track 0 or 1 dot. Implications for the use and design of redundant signals in multi-task contexts that vary in task demands are discussed.
Abstract: Zhang, Xijuan -- Keywords: Missing Data; Incomplete Data; Simulation Studies; Creating Missing Data; Generating Missing Data. -- Abstract : Missing data are common in psychological and educational research. With the improvement in computing technology in recent decades, more researchers have begun developing missing data techniques. In their research, they often conduct Monte Carlo simulation studies to compare the performances of different missing data techniques. During such simulation studies, researchers must generate missing data in the simulated dataset by deciding which data values to delete. However, in the current literature, there are limited guidelines on how to generate missing data for simulation studies. Our paper is one of the first that examines ways of generating missing data for simulation studies. I emphasize the importance of specifying missing data rules which are statistical models for generating missing data. I begin the paper by reviewing the types of missing data mechanisms and missing data patterns. I then explain how to specify missing data rules to generate missing data with different mechanisms and patterns. I emphasize the advantages and disadvantages of using different missing data rules and algorithms to generate missing data for simulation studies. Next, I discuss other important aspects of simulation studies involving missing data. I end the paper by offering recommendations for generating missing data for simulation studies.
Abstract: Caron-Diotte, Mathieu et al. -- Keywords: missing data, unplanned missingness, planned missingness, full information maximum likelihood, multiple imputation.. -- Abstract : While analyzing data, researchers are often faced with missing values. This is especially common in longitudinal studies in which participants might skip assessments. Unwanted missing data can introduce bias in the results and should thus be handled appropriately. However, researchers can sometimes want to include missing values in their data collection design to reduce its length and cost, a method called ``planned missingness.'' This paper review the recommended practices for handling both planned and unplanned missing data, with a focus on longitudinal studies. The current guidelines suggest to either use Full Information Maximum Likelihood or Multiple Imputation. Those techniques are illustrated with R code in the context of a longitudinal study with a representative Canadian sample on the psychological impacts of the COVID-19 pandemic.
Abstract: Bradley-Garcia, Meenakshie et al. -- Keywords: n-back, working memory, cognition, neurocognitive tests, computer-based psychological testing. -- Abstract : The n-back task is an extensively used cognitive test that assesses working memory. The task is well-suited to virtual administration as it reliably produces similar results to in-person administration and is easily adapted to asynchronous operation. However, the procedure to program this task into various computer programs and software is not widely known. Therefore, this tutorial aims to provide researchers with simple yet detailed step-by-step instructions on how to program an n-back task in Qualtrics using HTML and JavaScript. This tutorial is meant to be easily followed by the layperson without extensive knowledge of computer programming.
Abstract: Dahm, Stephan F. et al. -- Keywords: repeated testing, experimental application tools, serial reaction time task. -- Abstract : Here, we provide tips and tricks for running multisession experiments out of the lab using OpenSesame, a user-friendly experimental tool that is open source and runs on Windows, MacOS, and Linux. We focus on learning experiments that involve the measurement of reaction times. We show how such experiments can be run with traditional desktop-based experiment software on participants’ own notebooks (i.e., out-of-the-lab, but not in a browser). Learning experiments pose specific challenges: accessing individual identifying numbers, accessing session numbers, and counterbalancing conditions across participants. This article includes helpful code and provides hands-on implementation tips that will be useful also beyond the presented use case. The aim of this article is to illustrate how to create multisession learning experiments even with little technical expertise. We conclude that, if done right, out-of-the-lab experiments are a valid alternative to traditional lab testing.
Abstract: Zhao, Xiang -- Keywords: visualization; latent class analysis; subgroup. -- Abstract : While latent class analysis (LCA) has gained popularity in social sciences, including psychology, the visualization of latent classes has been the subject of limited discussions. This article reviews the elements of LCA figures, covering issues such as graph type, axis labels, and subgroup naming. Bar charts and line graphs have been identified as two major visualization approaches in LCA studies. It is concluded that LCA figures serve as an important visual vehicle to display subgroup characteristics. However, the elements of LCA figures need careful consideration as they could furnish the text with additional information. A checklist is summarized for LCA figure preparation.
Abstract: Laurencelle, Louis et al. -- Keywords: frequency; contingency table; analyses of frequencies; additive decomposition. -- Abstract : Analyses of frequencies are commonly done using a chi-square test. This test, derived from a normal approximation, is deemed generally efficient (controlling type-I error rates fairly well and having good statistical power). However, in the case of factorial designs, it is difficult to decompose a total test statistic into additive interaction effects and main effects. Herein, we present an alternative test based on the $G$ statistic. The test has similar type-I error rates and power as the former one. However, it is based on a total statistic that is naturally decomposed additively into interaction effects, main effects, simple effects, contrast effects, etc., mimicking precisely the logic of ANOVAs. We call this set of tools ANOFA (Analysis of Frequency data) to highlight its similarities with ANOVA. We also examine how to render plots of frequencies along with confidence intervals. Finally, quantifying effect sizes and planning statistical power are described under this framework. The ANOFA is a tool that assesses the significance of effects instead of the significance of parameters; as such, it is more intuitive to most researchers than alternative approaches based on generalized linear models.
Abstract: Padilla, Miguel A. -- Keywords: Put keywords here, in a comma separated list. -- Abstract : Four different bootstrap methods for estimating confidence intervals (CIs) for a coefficient alpha difference from two independent samples (groups) were examined. These four CIs were compared to the most promising non-bootstrap CI alternatives in the literature. All CIs were assessed with a Monte Carlo simulation with conditions similar to previous research. The results indicate that there is a clear order in coverage performance of the CIs. The bootstrapped highest density interval had the best coverage performance across all simulation conditions. Yet, it was impacted by unequal sample sizes when one of the groups had the smallest sample size investigated of 50, or when items came from a compound symmetric correlation matrix with $\rho = 0.64$. Regardless of the simulation condition, the percentile bootstrap is a good alternative as long as both group sample sizes were 200 or more.
Abstract: Bellemare-Lepage, Agathe et al. -- Keywords: classes latentes, package poLCA, modélisation. -- Abstract : L'analyse de classes latentes (ACL) permet de partager et de distinguer des sous-groupes non observables (latents) d'individus sur la base de leurs réponses à un ensemble d'indicateurs observables (manifestes). Cette analyse permet de mieux comprendre la variabilité au sein d'une population. Or, il existe peu de documentation, surtout en français, sur la procédure à suivre pour réaliser une ACL sur la plateforme R. Ce logiciel statistique est accessible gratuitement et comporte de nombreux avantages en ce qui a trait à la programmation d'analyses, à la visualisation des données ainsi qu'à la gestion des variables et de l'environnement de travail. L'objectif du présent article est d'exemplifier la réalisation d'une ACL sur la plateforme R avec le package poLCA. Après une introduction sur l'origine et les principes de l'ACL, un tutoriel sur la réalisation d'une ACL avec R est présenté. Une situation hypothétique portant sur la perpétration de violence dans les relations amoureuses à l'adolescence est utilisée. La syntaxe R permettant de réaliser cette analyse est fournie et explicitée en détails. Dans une visée de partage des connaissances, similaire à la philosophie de R, cet article peut servir de guide pour tout étudiant ou chercheur voulant développer sa compréhension de l'ACL et ses compétences en statistiques sur cette plateforme.
Open Access journal
