JournalTOCs

About Help News

Methods, Data, Analyses
Number of Followers: 1

This is an Open Access Journal

Open Access journal
ISSN (Print) 1864-6956 - ISSN (Online) 2190-4936
Published by GESIS - Leibniz Institute for the Social Sciences

[1 journal]

Different Approaches to Incorporate the Aspect of Practical Relevance in
the Statistical Inferential Process
- Authors: Andreas Quatember
  First page: 9
  Abstract: In different scientific areas, empirical studies are typically carried out by statistical null hypothesis tests. Despite the long tradition of applications, misinterpretations and misuses of the concept have led to a substantial confidence crisis in its inferential quality. One of the discussed issues is the significance-relevance discrepancy of the results of standardly applied zero-effect null hypothesis tests. This means that statistically significant test results do not automatically also have to be of scientific relevance in the specific research context. Therefore, this article is aimed at practitioners of empirical research who might want to include the aspect of practical relevance in their statistical conclusions. Different approaches to include this aspect in the inferential process are discussed with an example from the field of educational research.
  PubDate: 2023-01-11
  DOI: 10.12758/mda.2022.07
  Issue No: Vol. 17, No. 1 (2023)
Evaluating an Alternative Frame for Address-Based Sampling in Germany: The
Address Database From Deutsche Post Direkt
- Authors: Sven Stadtmüller, Henning Silber, Tobias Gummer, Matthias Sand, Stefan Zins, Christoph Beuthner, Pablo Christmann
  First page: 17
  Abstract: In Germany, the population registers with addresses of individuals can be used for address-based sampling. However, unlike countries with a centralized register, municipalities in Germany administer their registers themselves. This not only makes sampling for a nationwide survey more costly and cumbersome but may also result in gaps in the gross sample, as selected municipalities may refuse to allow their registers to be used for sampling purposes. If substitute municipalities are not available, other sampling methods are required. The present study tested the feasibility of using the address database from Deutsche Post Direkt (ADB-DPD) as an alternative frame for address-based sampling in Germany. We simultaneously conducted two almost identical surveys in the German city of Mannheim with gross samples of equal size (N = 3,000). One sample was drawn from the city’s population register, the other from the commercial ADB-DPD. Our findings suggest that the ADB-DPD performs well both in terms of survey response and up-to-dateness. Due to relatively low costs and the fast provision of addresses, the ADB-DPD could be particularly attractive for survey projects with limited budgets and tight schedules. However, these benefits come at considerable cost. First, the use of the ADB-DPD is limited to self-administered surveys. More importantly, in the net sample of the DPD survey, women and young persons were considerably underrepresented. This indicates coverage issues about which DPD provided no further information. Based on our analyses, we offer practical insights into the feasibility of using the ADB-DPD for sampling purposes and suggest avenues for future research.
  PubDate: 2023-01-11
  DOI: 10.12758/mda.2022.06
  Issue No: Vol. 17, No. 1 (2023)
Measuring Congruence Between Voters and Parties in Online Surveys: Does
Question Wording Matter'
- Authors: Bastiaan Bruinsma
  First page: 21
  Abstract: Congruence on policies between political parties and voters is a frequently assumed requirement for democracy. To be able to study this, we should be able to calculate accurate and precise measures of policy congruence in political systems. This could then tell us more about the political system we study, and the “distances” that exist between parties and voters on either issues or broader ideological dimensions. Here, I draw on experimental data from a Voting Advice Application to show that the wording of the issues can influence the degree of congruence one measures. Yet, this comes with the complication that this influence depends on the type of issue, the characteristics of the voters themselves, and the party the congruence is calculated with. These findings should serve as a warning for those who aim to measure congruence that even minor changes in question-wording can (but do not have to) cause relatively large changes in congruence, especially when many parties are involved and the differences between the congruences are small.
  PubDate: 2023-01-11
  DOI: 10.12758/mda.2022.12
  Issue No: Vol. 17, No. 1 (2023)
How to Optimize Online Mixed-Device Surveys: The Effects of a Messenger
Survey, Answer Scales, Devices and Personal Characteristics
- Authors: Caroline Marjanne Menken, Vera Toepoel
  First page: 24
  Abstract: The goal of this research was to determine the best way to present mixed-device surveys. We investigate the effect of survey method (messenger versus regular survey), answer scale, device used, and personal characteristics such as gender, age and education on break-off rate, substantive answers, completion time and respondents’ evaluation of the survey. Our research does not suggest that a messenger survey affects mixed-device surveys positively. Further research is necessary to investigate how to optimally present mixed-device surveys in order to increase participation and data quality.
  PubDate: 2023-01-11
  DOI: 10.12758/mda.2022.08
  Issue No: Vol. 17, No. 1 (2023)
River Sampling – a Fishing Expedition: A Non-Probability Case Study
- Authors: Alexander Murray-Watters, Stefan Zins, Henning Silber, Tobias Gummer, Clemens M. Lechner
  First page: 25
  Abstract: The ease with which large amounts of data can be collected via the Internet has led to a renewed interest in the use of non-probability samples. To that end, this paper performs a case study, comparing two non-probability datasets – one based on a river-sampling approach, one drawn from an online-access panel – to a reference probability sample. Of particular interest is the single-question river-sampling approach, as the data collected for this study presents an attempt to field a multi-item scale with such a sampling method. Each dataset consists of the same psychometric measures for two of the Big-5 personality traits, which are expected to perform independently of sample composition. To assess the similarity of the three datasets we compare their correlation matrices, apply linear and non-linear dimension reduction techniques, and analyze the distance between the datasets. Our results show that there are important limitations when implementing a multi-item scale via a single-question river sample. We find that, while the correlation between our data sets is similar, the samples are composed of persons with different personality traits.
  PubDate: 2023-01-11
  DOI: 10.12758/mda.2022.05
  Issue No: Vol. 17, No. 1 (2023)
Do We Have to Mix Modes in Probability-Based Online Panel Research to
Obtain More Accurate Results'
- Authors: Sebastian Kocar, Nicholas Biddle
  First page: 27
  Abstract: Online probability-based panels often apply two or more data collection modes to cover both the online and offline populations with the aim of obtaining results that are more representative of the population of interest. This study used such a panel to investigate how necessary it is, from the coverage error standpoint, to include the offline population by mixing modes in online panel survey research. This study evaluated the problem from three different perspectives: undercoverage bias, bias related to survey item topics and variable characteristics, and accuracy of online-only samples relative to nationally representative benchmarks. The results indicated that attitudinal, behavioral, and factual differences between the online and offline populations in Australia are, on average, minor. This means that, considering that survey research commonly includes a relatively low proportion of the offline population, survey estimates would not be significantly affected if probability-based panels did not mix modes and instead were online only, for the majority of topics. The benchmarking analysis showed that mixing the online mode with the offline mode did not improve the average accuracy of estimates relative to nationally representative benchmarks. Based on these findings, it is argued that other online panels should study this issue from different perspectives using the approaches proposed in this paper. There might also be an argument for (temporarily) excluding the offline population in probability-based online panel research in particular country contexts as this might have practical implications.
  PubDate: 2023-01-11
  DOI: 10.12758/mda.2022.11
  Issue No: Vol. 17, No. 1 (2023)

JournalTOCs
School of Mathematical and Computer Sciences
Heriot-Watt University
Edinburgh, EH14 4AS, UK
Email: journaltocs@hw.ac.uk
Tel: +00 44 (0)131 4513762

Your IP address: 44.192.38.49

Home (Search)
API
About JournalTOCs
News (blog, publications)