|
|
- Addressing Dependency in Meta-Analysis: A Companion to Assink and
Wibbelink (2016)
Abstract: Assink, Mark et al. -- Keywords: meta-analysis, three-level meta-analysis, multivariate meta-analysis, robust variance estimation, effect size dependency. -- Abstract : This research note elaborates on addressing dependency in effect size data and serves as a companion to our tutorial on fitting three-level meta-analytic models in R (Assink and Wibbelink, 2016). We provide a description of effect size and standard error dependency, explain how both the multilevel and multivariate meta-analytic models handle these types of dependency, and discuss the role of alternative methods in addressing dependency in effect size data, including approximating a variance-covariance matrix and applying a cluster-robust inference method. These alternative methods are illustrated with example R code that builds upon the effect size dataset that we presented and analyzed in our tutorial. We conclude that more simulation studies are needed to provide clearer guidelines for modeling dependency in effect size data and urge statisticians to make the available technical literature further accessible to applied researchers.
- Methodology for Identification, Visualization, and Clustering of Similar
Behaviors in Dyadic Sequences Analyzed Through the Longitudinal
Actor-Partner Interdependence Model With Markov Chains
Abstract: Bollenrücher, Mégane et al. -- Keywords: Dyadic sequence; APIM model; Markov chains. -- Abstract : The longitudinal actor-partner interdependence model (L-APIM) is frequently used to study dyadic relationships over time. When one deals with categorical longitudinal data, Markov chains emerge as a valuable analytical tool. This approach allows for the identification of interaction patterns in the L-APIM framework through the examination of the transition matrix. In the context of dyadic sample, investigating the similarity of behaviors between individuals becomes important. To address this question, visualization and grouping analysis are employed, providing valuable tools for discerning relationships with behavioral data. We introduce a novel methodological approach to ascertain such behavioral similarity using the probabilities into the transition matrix. In this article, we describe the utilization of multidimensional scaling and hierarchical clustering for identifying analogous behaviors within a dyadic sample. We illustrate the complete methodology using a simulated dataset. Codes in R language are included for implementation.
- Evaluating Assessment via Item Response Theory Utilizing Information
Function With R
Abstract: Tan, Teck Kiang -- Keywords: Test information function, Item information function, Item response theory. -- Abstract : Item and test information functions that measure the reliability of an assessment via Item response theory (IRT) are described in the paper for the practitioners. While the four parameters binary IRT models are frequently used, their relationship to the precision level is not commonly discussed. More notably, the benefits, limitations, and constraints of the information approach have not been fully examined systematically. On this basis, with useful and practical examples, the paper formally introduces the graphical approach of presenting item and test information functions that could be easily carried out using the irt R package. The simple R syntax is illuminated throughout the text to show the separate item parameter effects and the combinational and offsetting effects on the information when all the item parameters are used in an assessment. In particular, the characteristics of the 4PL information function that have not been paid much attention to are highlighted and illustrated about its functionality and application. The paper ends with a guide on the information approach to the selection of the items and setting up an assessment. The scope, limitations, and constraints of the graphical approach are also discussed.
- ICO, un indice de la consistance ordinale d’une série
statistique
Abstract: Laurencelle, Louis -- Keywords: Ordinal consistency, monotonicity; data order. -- Abstract : Une série numérique générée à partir d’un processus séquentiel montre-t-elle une tendance identifiable (monotonicité partielle ou complète), un désordre excessif, ou une simple variance d’erreur' L’indice ICO développé ici et inspiré des travaux de von Neumann (1941) sur la variance permutative répond à cette question. Formules, exemples et valeurs critiques sont inclus. // Does a numerical series generated from a sequential process show an identifiable trend (complete or segmental monotonicity), excessive disorder, or plain error variation' The IOC index developed, inspired by the work of von Neumann (1941) in ``The mean square successive difference'', answers this question. Formulas, examples and critical values are included.
- A Compendium of Common Heuristics, Misconceptions, and Biased Reasoning
used in Statistical Thinking
Abstract: Husereau, Tracy et al. -- Keywords: Statistics reasoning; heuristics; cognitive biases; misconceptions. -- Abstract : Over the past decades, many researchers have identified ways of reasoning in the domain of statistics and probabilities that do not match statistics and probabilities results. Some of these inadequate conceptualizations are reviewed herein. They include, among others, the gambler’s fallacy, the law of small numbers, the misunderstanding of randomness, and they touch various aspects of statistics reasoning (sampling procedures, probability estimation, mean estimation, variance estimation, and inference). A classification is put forward.
- Dissecting the Onion: Identifying and Remediating Issues Surrounding Data
Integrity in Online Survey Research
Abstract: Modrakovic, Xen et al. -- Keywords: Online Surveys, Data Integrity, Survey Development, Respondent Incentives, Bots. -- Abstract : In this non-empirical article, which is intended as a decision-making resource for researchers, we identify issues surrounding data integrity that commonly arise in online survey research and we propose remediation strategies based on challenges we encountered during a particular pilot study as well as our collective experience with conducting online survey research. Using the metaphor of an onion, we peel off the layers of this complex problem, synthesize the various available strategies used across disciplines, and propose some novel ones based on our perspective as psychologists. Corresponding to this multi-layered problem, we propose multi-layered solutions to prevent illegitimate responding—by both humans and non-humans (robots or “bots” for short)—from compromising the quality of data collected via online survey research. The first layer entails strategic item selection and protective programming in survey development. The second layer involves astute advertising and recruitment tactics to minimize illegitimate responses during survey dissemination. The third layer includes algorithms and other mechanisms to identify suspicious responses for possible exclusion during data verification. When we peel off the layers and reach the core problem of illegitimate responses to online surveys—financial incentives—we will propose ways of navigating respondent reimbursement to mitigate their inadvertent harmful impacts on the research process. By proposing these solutions, we aim to protect the integrity of scientific inquiry in psychology, especially given how often this method is used in the discipline.
- La temporalité dans les analyses de médiation
Abstract: Vivier, Olivier et al. -- Keywords: Médiation, CLPM, meilleures pratiques, temporalité, devis transversal. -- Abstract : L’analyse de médiation permet d’explorer un processus qui se déploie de façon séquentielle dans le temps. Toutefois, négliger la séquence temporelle entre les variables peut mener à d’importants biais statistiques. Afin d’éviter des problèmes méthodologiques et statistiques qui mènent à des conclusions erronées, il est crucial de connaître les spécificités inhérentes à l’analyse de médiation. Cet article décrit les limites courantes découlant des analyses de médiation et propose des solutions. Notamment, les analyses transversales ne sont pas adéquates, car un effet indirect correspond davantage à un effet temporel. Quant aux modèles longitudinaux, ceux-ci peuvent mener à des biais si les effets autorégressifs ou les effets intra et inter-sujets sont négligés. L’utilisation de solutions plus adéquates comme les courbes de croissances latentes et les modèles multiniveaux s’avèrent un choix judicieux pour examiner des effets indirects lorsque des données sont collectées à au moins trois temps de mesure.
- Identifying Influential Observations in Multiple Regression
Abstract: Camilleri, Carmel et al. -- Keywords: Influential Cases, Monte Carlo Simulation, Outliers, Cook’s Distance, DFFITS. -- Abstract : Linear models are particularly vulnerable to influential observations which disproportionately affect the model's parameter estimates. Multiple statistics and numerous cut-off values have been proposed to detect highly influential observations including Cook’s Distance (CD), Standardized Difference of Fits (DFFITS) and Standardized Difference of Beta (DFBETAS). This paper reports on a Monte Carlo simulation study that assesses the effectiveness of these methods and recommended cut-off values under various conditions, including different sample sizes, numbers of predictors, strengths of variable associations, and non-sequential versus sequential analysis approaches within a multiple linear regression framework. The findings suggest that the proportion of observations identified as highly influential varies significantly based on the chosen diagnostic method and the thresholds used for detection. Consequently, researchers should consider the implications of their methodological choices and the thresholds they apply when identifying influential data points.
- Response Time Distribution Analysis of Medium-Sized Datasets in MATLAB
Abstract: Blinch, Jarrod et al. -- Keywords: ex-Gaussian distribution function, geometric quantile averaging (GQA), quantile maximum probability estimator (QMPE). -- Abstract : Response time data have a positively skewed distribution. The challenge with this is that a measure of central tendency and dispersion does not adequately describe a skewed distribution. A researcher relying on only response time mean and standard deviation could make incorrect conclusions about response time. The best way to analyze response time data is with a distribution analysis. One reason that response time distribution analyses are atypical is that at least 100 trials are recommended per participant and condition. In the current tutorial, we demonstrate a distribution analysis technique that requires as few as 40 participants with 40 trials per condition. This technique involves geometric quantile averaging (GQA) and the quantile maximum probability estimator (QMPE). Each step of the analysis is detailed with a MATLAB script, flexible MATLAB functions, and experimental response time data. Our goal was to lower the barriers to entry for response time distribution analysis so that more researchers will choose to thoroughly examine response time data.
- Exploring dynamic structures of dyadic conversations using categorical
cross recurrence quantification analysis: A tutorial
Abstract: Duong, Shirley et al. -- Keywords: Dyadic interactions, categorical data, cross recurrence quantification. -- Abstract : Social interactions are defined by the dynamic and reciprocal exchange of information in a process referred to as mutual alignment. Statistical methods for characterizing alignment between two interacting partners are emerging. In general, they exploit the temporal organization of dyadic interactions to uncover the effect of one partner on the other and the extent to which partners are aligned. This paper describes and provides a tutorial on one such method, categorical cross recurrence quantification analysis (CRQA), which quantifies the temporal structure and co-visitation of individual and sequential states of interest. CRQA is a useful descriptive technique that can be used to explore the extent, structures, and patterns of partner alignment within dyadic interactions. We provide a brief technical introduction to CRQA and a tutorial on its application to understanding parent-child linguistic interactions using the ‘crqa’ package in R (Coco, Monster, Leonardi, Dale, & Wallot, 2021).
- Dynamic structures of parent-child number talk: An application of
categorical cross-recurrence quantification analysis and companion to
Duong et al. (2024)
Abstract: Duong, Shirley et al. -- Keywords: parent-child interactions, number talk, math skills, recurrence quantification analysis. -- Abstract : Social interactions, particularly parent-child conversations, play a critical role in children’s early learning and pre-academic skill development. While these interactions are bidirectional, complex, and dynamic, much of the research in this area tends to separate speakers’ talk and capture the frequency of words or utterances. Beyond the aggregation of talk exists rich information about conversational structures and processes, such as the extent to which speakers are aligned or reciprocate each other’s talk. These measures can be derived using categorical cross-recurrence quantification analysis (CRQA), a method that quantifies the temporal structure and co-visitation of individual and sequential events, e.g., utterances between speakers. In this paper, we present an application of CRQA, following the protocol described in our tutorial paper (Duong et al., 2024, this issue), to describe alignment in parent-child conversations about numbers and math (i.e., number talk). We used the ‘crqa’ package in R and the code used in this application is available in the Supplemental Materials. Further, the CRQA measures derived from this application were compared to traditional frequency measures of talk, i.e., counts of utterances, in the prediction of children’s math skills. Overall, we showed that (1) CRQA can be applied to existing transcription data to uncover theoretically-driven patterns of parent-child talk that are not captured by common frequency measures and (2) these CRQA measures offer additional, rich information about interactions beyond frequencies of talk and can be used to predict individual differences in children’s math skills.
- Compte rendu du colloque "Méthodes Quantitatives en Sciences Humaines
(MQSH) 2024"
Abstract: Caron, Pier-Olivier et al. -- Keywords: Statistiques, Actes de colloque, MQSH 2024. -- Abstract : Le vendredi 7 juin 2024 s’est tenu le 13e colloque annuel Méthodes quantitatives en sciences humaines (MQSH) à l’Université TÉLUQ à Montréal. Cinq conférenciers et conférencières ont présenté leurs travaux dans le domaine. Eric Frenette a discuté de l’évolution du modèle Bifacteur, depuis sa proposition par Holzinger, suite aux travaux de Spearman en 1927, jusqu’à sa version contemporaine, version digne du monstre de Frankenstein, qu’elle est devenue en y transfigurant ses hypothèses sous-jacentes et en y rapiéçant des paramètres augmentant son potentiel explicatif. Sophie Vanbelle, conférencière invitée pour l’évènement, a exposé les principales mesures statistiques et leur interprétation pour déterminer la fiabilité et l’accord dans le cas d’une échelle binaire. Elle a discuté de l’importance de rapporter les différents indices d’accord, de calculer l’effectif d’échantillon nécessaire pour avoir des mesures fiables de l’accord interjuges (et intrajuges) et du besoin de lignes directrices pour les chercheurs et chercheuses. André Achim a proposé une approche novatrice de l’analyse factorielle exploratoire reposant non pas sur la décomposition de matrices, mais plutôt sur l’annulation du signal, ce qui permet d’extraire toutes les informations pertinentes, le nombre de facteurs et la structure factorielle, tout en réduisant le nombre d’options décisionnelles. Ibtissem Ben Alaya a présenté le processus de validation en sept étapes d’un questionnaire portant sur les compétences socioémotionnelles des enseignants et enseignantes tout en discutant des défis rencontrés ainsi que des choix psychométriques pris. Louis Laurencelle, dans sa présentation, a critiqué le raisonnement et les critères d’altération du taux d’erreur de type I (seuil alpha), telles la correction de Bonferroni et celle de Tukey, lesquels torpillent la puissance statistique des travaux de recherche en sciences humaines. Il a examiné et argumenté la nocivité d’une telle approche et son absurdité. En conclusion, cette 13e édition du colloque MQSH fut digne de sa bannière : compétence, critique et créativité.
- Relative importance analysis for count regression models
Abstract: Luchman, Joseph N. -- Keywords: Dominance Analysis, Relative Importance, Poisson Regression, R-square, Negative Binomial Regression. -- Abstract : Count variables are common in behavioral science as an outcome. Count regression models, such as Poisson regression, are recommended when analyzing count variables but can be challenging to interpret given their non-linear functional form. I recommend relative importance analysis as a method to use in interpreting count regression model results. This work extends on past research by describing an approach to determining the importance of independent variables in count regression models using dominance analysis. Herein, dominance analysis is reviewed as a relative importance method, recommend a pseudo-$R^2$ to use with count regression model-based dominance analysis, and outline the results of an analysis with simulated data that uses the recommended methodology. This work contributes to the literature by extending dominance analysis to count regression models and provides a thoroughly documented example analysis that researchers can use to implement the methodology in their research.
- Local decorrelation for error bars in time series
Abstract: Cousineau, Denis et al. -- Keywords: Error bars, confidence intervals, time series, ERP, EEG. -- Abstract : Time series and electroencephalographic data are often noisy sources of data. In addition, the samples are often small or medium so that confidence intervals for a given time point taken in isolation may be large. Decorrelation techniques were shown to be adequate and exact for repeated-measure designs where correlation is assumed constant across pairs of measurements. This assumption cannot be assumed in time series and electroencephalographic data where correlations are most-likely vanishing with temporal distance between pairs of points. Herein, we present a decorrelation technique based on an assumption of local correlation. This technique is illustrated with fMRI data from 14 participants and from EEG data from 24 participants.
- From Scratch: A Didactic Example of Multivariate Regression in Python
Abstract: Ashiabi, Godwin S. -- Keywords: linear hypothesis testing, multiple regression, multivariate regression. -- Abstract : Although Python has the statsmodels library that can be used to perform different statistical analyses, including multiple regression, it has not yet implemented multivariate regression as one of its methods. An investigator interested in conducting a multivariate regression is thus limited to running a series of univariate regression models, which do not take into account the collinearity among variables in the models. Thus, the purpose of this tutorial is to demonstrate how to perform multivariate regression in Python using custom user-defined classes, and linear hypothesis testing using statsmodels.
- Articulatory rehearsal and phonological storage in working memory: A
replication of Longoni et al. (1993)
Abstract: Couture, Camille et al. -- Keywords: phonological loop replication, word length effect, phonemic similarity, working memory, replicability predicament. -- Abstract : This study sought to replicate Longoni et al.'s (1993) investigation into the multiple subsystems within the phonological loop, originally proposed by Baddeley et al. (1984). To address the replicability crisis in scientific research, this replication maintains similar methodologies but also features a more diverse sample, characterized solely by advanced French proficiency and increased participant numbers. In addition, this study examined various characteristics of the phonological loop, including the phonemic similarity effect, word length effect, presentation conditions, articulatory suppression effect, and irrelevant sounds. However, the articulatory suppression effect, word length effect, presentation speed effect, and delay effect did not replicate. Notably, an interaction emerged between word length and phonemic similarity, indicating a multiplicative relationship rather than an additive one. The present study emphasizes the importance of further exploration into the factors that influence the non-replicated effects, including the discriminant validity of the variables. Future studies could leverage insights from the phonological loop's characteristics to enhance understanding of the working memory system.
|
Open Access journal
