Authors:Shaobo Jin; Chunzheng Cao Abstract: The polychoric instrumental variable (PIV) approach is a recently proposed method to fit a confirmatory factor analysis model with ordinal data. In this paper, we first examine the small-sample properties of the specification tests for testing the validity of instrumental variables (IVs). Second, we investigate the effects of using different numbers of IVs. Our results show that specification tests derived for continuous data are extremely oversized at all sample sizes when applied to ordinal variables. Possible modifications for ordinal data are proposed in the present study. Simulation results show that the modified specification tests with all available IVs are able to detect model misspecification. In terms of estimation accuracy, the PIV approach where the IVs outnumber the endogenous variables by one produces a lower bias but a higher variation than the PIV approach with more IVs for correctly specified factor loadings at small samples. PubDate: 2018-01-11T07:35:28.366799-05: DOI: 10.1111/bmsp.12128

Authors:Oscar L. Olvera Astivia; Bruno D. Zumbo Abstract: The Fleishman third-order polynomial algorithm is one of the most-often used non-normal data-generating methods in Monte Carlo simulations. At the crux of the Fleishman method is the solution of a non-linear system of equations needed to obtain the constants to transform data from normality to non-normality. A rarely acknowledged fact in the literature is that the solution to this system is not unique, and it is currently unknown what influence the different types of solutions have on the computer-generated data. To address this issue, analytical and empirical investigations were conducted, aimed at documenting the impact that each solution type has on the design of computer simulations. In the first study, it was found that certain types of solutions generate data with different multivariate properties and wider coverage of the theoretical range spanned by population correlations. In the second study, it was found that previously published recommendations from Monte Carlo simulations could change if different types of solutions were used to generate the data. A mathematical description of the multiple solutions to the Fleishman polynomials is provided, as well as recommendations for users of this method. PubDate: 2018-01-11T07:30:24.211911-05: DOI: 10.1111/bmsp.12126

Authors:R. Philip Chalmers Abstract: An efficient and accurate numerical approximation methodology useful for obtaining the observed information matrix and subsequent asymptotic covariance matrix when fitting models with the EM algorithm is presented. The numerical approximation approach is compared to existing algorithms intended for the same purpose, and the computational benefits and accuracy of this new approach are highlighted. Instructive and real-world examples are included to demonstrate the methodology concretely, properties of the estimator are discussed in detail, and a Monte Carlo simulation study is included to investigate the behaviour of a multi-parameter item response theory model using three competing finite-difference algorithms. PubDate: 2018-01-09T03:30:33.65625-05:0 DOI: 10.1111/bmsp.12127

Authors:Sandip Sinharay Abstract: Tatsuoka suggested several extended caution indices and their standardized versions, and these have been used as person-fit statistics by various researchers. However, these indices are only defined for tests with dichotomous items. This paper extends two of the popular standardized extended caution indices for use with polytomous items and mixed-format tests. Two additional new person-fit statistics are obtained by applying the asymptotic standardization of person-fit statistics for mixed-format tests. Detailed simulations are then performed to compute the Type I error rate and power of the four new person-fit statistics. Two real data illustrations follow. The new person-fit statistics appear to be satisfactory tools for assessing person fit for polytomous items and mixed-format tests. PubDate: 2018-01-09T03:25:40.765201-05: DOI: 10.1111/bmsp.12124

Authors:Douglas Steinley; Michael J. Brusco Abstract: Two expectations of the adjusted Rand index (ARI) are compared. It is shown that the expectation derived by Morey and Agresti (1984, Educational and Psychological Measurement, 44, 33) under the multinomial distribution to approximate the exact expectation from the hypergeometric distribution (Hubert & Arabie, 1985, Journal of Classification, 2, 193) provides a poor approximation, and, in some cases, the difference between the two expectations can increase with the sample size. Proofs concerning the minimum and maximum difference between the two expectations are provided, and it is shown through simulation that the ARI can differ significantly depending on which expectation is used. Furthermore, when compared in a hypothesis testing framework, multinomial approximation overly favours the null hypothesis. PubDate: 2017-11-20T23:20:52.50852-05:0 DOI: 10.1111/bmsp.12116

Authors:Hao Wu Abstract: In structural equation modelling (SEM), a robust adjustment to the test statistic or to its reference distribution is needed when its null distribution deviates from a χ2 distribution, which usually arises when data do not follow a multivariate normal distribution. Unfortunately, existing studies on this issue typically focus on only a few methods and neglect the majority of alternative methods in statistics. Existing simulation studies typically consider only non-normal distributions of data that either satisfy asymptotic robustness or lead to an asymptotic scaled χ2 distribution. In this work we conduct a comprehensive study that involves both typical methods in SEM and less well-known methods from the statistics literature. We also propose the use of several novel non-normal data distributions that are qualitatively different from the non-normal distributions widely used in existing studies. We found that several under-studied methods give the best performance under specific conditions, but the Satorra–Bentler method remains the most viable method for most situations. PubDate: 2017-10-31T01:20:31.842068-05: DOI: 10.1111/bmsp.12123

Authors:Shiyu Wang Abstract: The maximum likelihood classification rule is a standard method to classify examinee attribute profiles in cognitive diagnosis models (CDMs). Its asymptotic behaviour is well understood when the model is assumed to be correct, but has not been explored in the case of misspecified latent class models. This paper investigates the asymptotic behaviour of a two-stage maximum likelihood classifier under a misspecified CDM. The analysis is conducted in a general restricted latent class model framework addressing all types of CDMs. Sufficient conditions are proposed under which a consistent classification can be obtained by using a misspecified model. Discussions are also provided on the inconsistency of classification under certain model misspecification scenarios. Simulation studies and a real data application are conducted to illustrate these results. Our findings can provide some guidelines as to when a misspecified simple model or a general model can be used to provide a good classification result. PubDate: 2017-10-28T00:10:38.057974-05: DOI: 10.1111/bmsp.12119

Authors:Dylan Molenaar; Maria Bolsinova, Jeroen K. Vermunt Abstract: In item response theory, modelling the item response times in addition to the item responses may improve the detection of possible between- and within-subject differences in the process that resulted in the responses. For instance, if respondents rely on rapid guessing on some items but not on all, the joint distribution of the responses and response times will be a multivariate within-subject mixture distribution. Suitable parametric methods to detect these within-subject differences have been proposed. In these approaches, a distribution needs to be assumed for the within-class response times. In this paper, it is demonstrated that these parametric within-subject approaches may produce false positives and biased parameter estimates if the assumption concerning the response time distribution is violated. A semi-parametric approach is proposed which resorts to categorized response times. This approach is shown to hardly produce false positives and parameter bias. In addition, the semi-parametric approach results in approximately the same power as the parametric approach. PubDate: 2017-10-17T02:15:57.121266-05: DOI: 10.1111/bmsp.12117

Authors:Zijun Ke; Zhiyong (Johnny) Zhang Abstract: Autocorrelation and partial autocorrelation, which provide a mathematical tool to understand repeating patterns in time series data, are often used to facilitate the identification of model orders of time series models (e.g., moving average and autoregressive models). Asymptotic methods for testing autocorrelation and partial autocorrelation such as the 1/T approximation method and the Bartlett's formula method may fail in finite samples and are vulnerable to non-normality. Resampling techniques such as the moving block bootstrap and the surrogate data method are competitive alternatives. In this study, we use a Monte Carlo simulation study and a real data example to compare asymptotic methods with the aforementioned resampling techniques. For each resampling technique, we consider both the percentile method and the bias-corrected and accelerated method for interval construction. Simulation results show that the surrogate data method with percentile intervals yields better performance than the other methods. An R package pautocorr is used to carry out tests evaluated in this study. PubDate: 2017-09-12T06:20:37.586728-05: DOI: 10.1111/bmsp.12109

Authors:Jenő Reiczigel; Márton Ispány, Gábor Tusnády, György Michaletzky, Marco Marozzi Abstract: Rudas, Clogg, and Lindsay (1994, J. R Stat Soc. Ser. B, 56, 623) introduced the so-called mixture index of fit, also known as pi-star (π*), for quantifying the goodness of fit of a model. It is the lowest proportion of ‘contamination’ which, if removed from the population or from the sample, makes the fit of the model perfect. The mixture index of fit has been widely used in psychometric studies. We show that the asymptotic confidence limits proposed by Rudas et al. (1994, J. R Stat Soc. Ser. B, 56, 623) as well as the jackknife confidence interval by Dayton (, Br. J. Math. Stat. Psychol., 56, 1) perform poorly, and propose a new bias-corrected point estimate, a bootstrap test and confidence limits for pi-star. The proposed confidence limits have coverage probability much closer to the nominal level than the other methods do. We illustrate the usefulness of the proposed method in practice by presenting some practical applications to log-linear models for contingency tables. PubDate: 2017-09-12T05:30:27.127121-05: DOI: 10.1111/bmsp.12118

Authors:Wolfgang Wiedermann; Edgar C. Merkle, Alexander Eye Abstract: Methods to determine the direction of a regression line, that is, to determine the direction of dependence in reversible linear regression models (e.g., xy vs. yx), have experienced rapid development within the last decade. However, previous research largely rested on the assumption that the true predictor is measured without measurement error. The present paper extends the direction dependence principle to measurement error models. First, we discuss asymmetric representations of the reliability coefficient in terms of higher moments of variables and the attenuation of skewness and excess kurtosis due to measurement error. Second, we identify conditions where direction dependence decisions are biased due to measurement error and suggest method of moments (MOM) estimation as a remedy. Third, we address data situations in which the true outcome exhibits both regression and measurement error, and propose a sensitivity analysis approach to determining the robustness of direction dependence decisions against unreliably measured outcomes. Monte Carlo simulations were performed to assess the performance of MOM-based direction dependence measures and their robustness to violated measurement error assumptions (i.e., non-independence and non-normality). An empirical example from subjective well-being research is presented. The plausibility of model assumptions and links to modern causal inference methods for observational data are discussed. PubDate: 2017-09-05T00:18:20.67104-05:0 DOI: 10.1111/bmsp.12111

Authors:Peida Zhan; Hong Jiao, Dandan Liao Abstract: To provide more refined diagnostic feedback with collateral information in item response times (RTs), this study proposed joint modelling of attributes and response speed using item responses and RTs simultaneously for cognitive diagnosis. For illustration, an extended deterministic input, noisy ‘and’ gate (DINA) model was proposed for joint modelling of responses and RTs. Model parameter estimation was explored using the Bayesian Markov chain Monte Carlo (MCMC) method. The PISA 2012 computer-based mathematics data were analysed first. These real data estimates were treated as true values in a subsequent simulation study. A follow-up simulation study with ideal testing conditions was conducted as well to further evaluate model parameter recovery. The results indicated that model parameters could be well recovered using the MCMC approach. Further, incorporating RTs into the DINA model would improve attribute and profile correct classification rates and result in more accurate and precise estimation of the model parameters. PubDate: 2017-09-05T00:00:50.512892-05: DOI: 10.1111/bmsp.12114

Authors:Anthony J. Bishara; Jiexiang Li, Thomas Nash Abstract: When bivariate normality is violated, the default confidence interval of the Pearson correlation can be inaccurate. Two new methods were developed based on the asymptotic sampling distribution of Fisher's z′ under the general case where bivariate normality need not be assumed. In Monte Carlo simulations, the most successful of these methods relied on the (Vale & Maurelli, 1983, Psychometrika, 48, 465) family to approximate a distribution via the marginal skewness and kurtosis of the sample data. In Simulation 1, this method provided more accurate confidence intervals of the correlation in non-normal data, at least as compared to no adjustment of the Fisher z′ interval, or to adjustment via the sample joint moments. In Simulation 2, this approximate distribution method performed favourably relative to common non-parametric bootstrap methods, but its performance was mixed relative to an observed imposed bootstrap and two other robust methods (PM1 and HC4). No method was completely satisfactory. An advantage of the approximate distribution method, though, is that it can be implemented even without access to raw data if sample skewness and kurtosis are reported, making the method particularly useful for meta-analysis. Supporting information includes R code. PubDate: 2017-09-04T23:56:02.899534-05: DOI: 10.1111/bmsp.12113

Authors:Nicolas Gauvrit; Fabien Mathy Abstract: The time-based resource sharing (TBRS) model is a prominent model of working memory that is both predictive and simple. TBRS is a mainstream decay-based model and the most susceptible to competition with interference-based models. A connectionist implementation of TBRS, TBRS*, has recently been developed. However, TBRS* is an enriched version of TBRS, making it difficult to test general characteristics resulting from TBRS assumptions. Here, we describe a novel model, TBRS2, built to be more transparent and simple than TBRS*. TBRS2 is minimalist and allows only a few parameters. It is a straightforward mathematical transcription of TBRS that focuses exclusively on the activation level of memory items as a function of time. Its simplicity makes it possible to derive several theorems from the original TBRS and allows several variants of the refresh process to be tested without relying on particular architectures. PubDate: 2017-09-04T23:46:02.56236-05:0 DOI: 10.1111/bmsp.12112

Authors:Jolien Cremers; Kees Tim Mulder, Irene Klugkist Abstract: The interpretation of the effect of predictors in projected normal regression models is not straight-forward. The main aim of this paper is to make this interpretation easier such that these models can be employed more readily by social scientific researchers. We introduce three new measures: the slope at the inflection point (bc), average slope (AS) and slope at mean (SAM) that help us assess the marginal effect of a predictor in a Bayesian projected normal regression model. The SAM or AS are preferably used in situations where the data for a specific predictor do not lie close to the inflection point of a circular regression curve. In this case bc is an unstable and extrapolated effect. In addition, we outline how the projected normal regression model allows us to distinguish between an effect on the mean and spread of a circular outcome variable. We call these types of effects location and accuracy effects, respectively. The performance of the three new measures and of the methods to distinguish between location and accuracy effects is investigated in a simulation study. We conclude that the new measures and methods to distinguish between accuracy and location effects work well in situations with a clear location effect. In situations where the location effect is not clearly distinguishable from an accuracy effect not all measures work equally well and we recommend the use of the SAM. PubDate: 2017-09-04T01:29:34.033252-05: DOI: 10.1111/bmsp.12108

Authors:Xin Gu; Joris Mulder, Herbert Hoijtink Abstract: Informative hypotheses are increasingly being used in psychological sciences because they adequately capture researchers’ theories and expectations. In the Bayesian framework, the evaluation of informative hypotheses often makes use of default Bayes factors such as the fractional Bayes factor. This paper approximates and adjusts the fractional Bayes factor such that it can be used to evaluate informative hypotheses in general statistical models. In the fractional Bayes factor a fraction parameter must be specified which controls the amount of information in the data used for specifying an implicit prior. The remaining fraction is used for testing the informative hypotheses. We discuss different choices of this parameter and present a scheme for setting it. Furthermore, a software package is described which computes the approximated adjusted fractional Bayes factor. Using this software package, psychological researchers can evaluate informative hypotheses by means of Bayes factors in an easy manner. Two empirical examples are used to illustrate the procedure. PubDate: 2017-08-31T05:17:50.56784-05:0 DOI: 10.1111/bmsp.12110

Authors:Guogen Shan; Charles Bernick, Sarah Banks Abstract: This research was motivated by a clinical trial design for a cognitive study. The pilot study was a matched-pairs design where some data are missing, specifically the missing data coming at the end of the study. Existing approaches to determine sample size are all based on asymptotic approaches (e.g., the generalized estimating equation (GEE) approach). When the sample size in a clinical trial is small to medium, these asymptotic approaches may not be appropriate for use due to the unsatisfactory Type I and II error rates. For this reason, we consider the exact unconditional approach to compute the sample size for a matched-pairs study with incomplete data. Recommendations are made for each possible missingness pattern by comparing the exact sample sizes based on three commonly used test statistics, with the existing sample size calculation based on the GEE approach. An example from a real surgeon-reviewers study is used to illustrate the application of the exact sample size calculation in study designs. PubDate: 2017-06-30T05:20:29.005688-05: DOI: 10.1111/bmsp.12107

Authors:Wolf Schwarz; Dennis Reike Abstract: Using a standard repeated measures model with arbitrary true score distribution and normal error variables, we present some fundamental closed-form results which explicitly indicate the conditions under which regression effects towards (RTM) and away from the mean are expected. Specifically, we show that for skewed and bimodal distributions many or even most cases will show a regression effect that is in expectation away from the mean, or that is not just towards but actually beyond the mean. We illustrate our results in quantitative detail with typical examples from experimental and biometric applications, which exhibit a clear regression away from the mean (‘egression from the mean’) signature. We aim not to repeal cautionary advice against potential RTM effects, but to present a balanced view of regression effects, based on a clear identification of the conditions governing the form that regression effects take in repeated measures designs. PubDate: 2017-06-30T05:00:32.429163-05: DOI: 10.1111/bmsp.12106

Authors:Maria Bolsinova; Jesper Tijmstra Abstract: By considering information about response time (RT) in addition to response accuracy (RA), joint models for RA and RT such as the hierarchical model (van der Linden, 2007) can improve the precision with which ability is estimated over models that only consider RA. The hierarchical model, however, assumes that only the person's speed is informative of ability. This assumption of conditional independence between RT and ability given speed may be violated in practice, and ignores collateral information about ability that may be present in the residual RTs. We propose a posterior predictive check for evaluating the assumption of conditional independence between RT and ability given speed. Furthermore, we propose an extension of the hierarchical model that contains cross-loadings between ability and RT, which enables one to take additional collateral information about ability into account beyond what is possible in the standard hierarchical model. A Bayesian estimation procedure is proposed for the model. Using simulation studies, the performance of the model is evaluated in terms of parameter recovery, and the possible gain in precision over the standard hierarchical model and an RA-only model is considered. The model is applied to data from a high-stakes educational test. PubDate: 2017-06-21T01:11:04.146383-05: DOI: 10.1111/bmsp.12104

Authors:Tamar Kennet-Cohen; Dvir Kleper, Elliot Turvall Abstract: A frequent topic of psychological research is the estimation of the correlation between two variables from a sample that underwent a selection process based on a third variable. Due to indirect range restriction, the sample correlation is a biased estimator of the population correlation, and a correction formula is used. In the past, bootstrap standard error and confidence intervals for the corrected correlations were examined with normal data. The present study proposes a large-sample estimate (an analytic method) for the standard error, and a corresponding confidence interval for the corrected correlation. Monte Carlo simulation studies involving both normal and non-normal data were conducted to examine the empirical performance of the bootstrap and analytic methods. Results indicated that with both normal and non-normal data, the bootstrap standard error and confidence interval were generally accurate across simulation conditions (restricted sample size, selection ratio, and population correlations) and outperformed estimates of the analytic method. However, with certain combinations of distribution type and model conditions, the analytic method has an advantage, offering reasonable estimates of the standard error and confidence interval without resorting to the bootstrap procedure's computer-intensive approach. We provide SAS code for the simulation studies. PubDate: 2017-06-20T02:27:38.723418-05: DOI: 10.1111/bmsp.12105

Authors:Yoosun Jamie Kim; Robert A. Cribbie Abstract: Valid use of the traditional independent samples ANOVA procedure requires that the population variances are equal. Previous research has investigated whether variance homogeneity tests, such as Levene's test, are satisfactory as gatekeepers for identifying when to use or not to use the ANOVA procedure. This research focuses on a novel homogeneity of variance test that incorporates an equivalence testing approach. Instead of testing the null hypothesis that the variances are equal against an alternative hypothesis that the variances are not equal, the equivalence-based test evaluates the null hypothesis that the difference in the variances falls outside or on the border of a predetermined interval against an alternative hypothesis that the difference in the variances falls within the predetermined interval. Thus, with the equivalence-based procedure, the alternative hypothesis is aligned with the research hypothesis (variance equality). A simulation study demonstrated that the equivalence-based test of population variance homogeneity is a better gatekeeper for the ANOVA than traditional homogeneity of variance tests. PubDate: 2017-06-01T01:50:35.69062-05:0 DOI: 10.1111/bmsp.12103