JournalTOCs

Journal of Survey Statistics and Methodology
Journal Prestige (SJR): 2.212

Citation Impact (citeScore): 1
Number of Followers: 5

Hybrid journal (It can contain Open Access articles)
ISSN (Print) 2325-0984 - ISSN (Online) 2325-0992
Published by Oxford University Press

[425 journals]

Recent Innovations and Advances in Mixed-Mode Surveys
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 507 - 531
  Abstract: AbstractThe use of mixed-mode surveys has grown rapidly in recent years, due to both technological advances and the COVID-19 pandemic. The increased mixing of modes (and the adoption of newer digital modes like web and text messaging) necessitates an evaluation of the impact of these newer designs on survey errors and costs, as well as new techniques for disaggregating and adjusting for nonresponse and measurement errors. This special issue highlights recent innovations, applications, and evaluations of mixed-mode survey designs and identifies areas where additional research is required.
  PubDate: Mon, 17 Jun 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae025
  Issue No: Vol. 12, No. 3 (2024)
Concurrent, Web-First, or Web-Only' How Different Mode Sequences Perform
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 532 - 557
  Abstract: AbstractDuring the COVID-19 pandemic, many survey programs switched to self-administered modes of data collection, often offering respondents both web-based and paper-based questionnaires. However, there is little guidance as to when to offer which mode, especially when the aim is to recruit participants for a panel survey. This study examines the effectiveness of different mode-sequence designs by analyzing an experiment implemented in the recruitment wave of the German panel study “Family Research and Demographic Analysis.” We randomly assigned 108,256 individuals aged 18–49 years to one of three mode-sequence-design conditions: concurrent, web-first including a paper-based questionnaire with the second reminder (web-first-2), and web-first including a paper-based questionnaire with the third reminder (web-first-3). A fourth, simulated group did not receive a paper-based questionnaire (web-only). We analyzed how different mode-sequence designs affected outcome rates, sample composition, response distributions, data quality, share of paper-based questionnaires, postage costs, number of postal mailings in the recruitment wave, and participation in the first regular wave. Our results show no differences in response distributions and small differences in sample composition across the four mode-sequence designs. As the web-first-2 and simulated web-only designs yielded comparatively good response and recruitment rates at reasonable costs, we recommend their use when surveying adults between 18 and 49 years old.
  PubDate: Sat, 09 Mar 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae008
  Issue No: Vol. 12, No. 3 (2024)
Sequential and Concurrent Mixed-Mode Designs: A Tailored Approach
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 558 - 577
  Abstract: AbstractDue to rising costs and declining response rates, surveys are increasingly moving from face-to-face interviewing to a self-administered mixed-mode design. Mixed-mode surveys can be conducted using a concurrent or a sequential design. A sequential design in which the web mode is offered first is a common strategy for mixed-mode surveys as it reduces survey costs. However, when deciding which mode choice sequence to use, sample balance should also be considered. One approach to achieving a balanced sample might be to tailor the sequence of the choice of modes, or the mode choice sequence. For this purpose, we use an indicator that assigns the sampled persons to the different mode choice sequences to minimize the variability of response probabilities. In this study, we compare the sample composition achieved with a concurrent and a sequential design. Additionally, we investigate whether indicator-based tailoring of the two mode choice sequences can improve sample composition. We implemented a randomized experiment in the 2021 German General Social Survey (ALLBUS), which surveyed the general population aged 18 and older in private households (N = 5,342) using a mixed-mode design (web and mail). In a first step, respondents were randomly assigned to a concurrent or a sequential design. We find that the two mode choice sequences lead to a similar sample composition. Next, we identify age as the best available single indicator of the variables known before the survey to tailor the mode choice sequence. Our analyses show that a tailored approach based on age improves the sample composition slightly.
  PubDate: Mon, 08 Apr 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae016
  Issue No: Vol. 12, No. 3 (2024)
Should We Offer Web, Paper, or Both' A Comparison of Single- and
Mixed-Response Mode Treatments in a Mail Survey
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 593 - 623
  Abstract: AbstractThis article leverages a five-treatment response mode experiment (paper-only, web-only, sequential web-mail, choice, and choice-plus [choice with a promised incentive for responding online]) that was conducted within a nationally representative survey. Because this survey’s sample was drawn from respondents to another nationally representative survey, we have rich frame data that includes multiple indicators of comfort using the internet for our sample members and we can compare their response behavior across two surveys. We find that the paper-only treatment yielded a lower response rate than most of the other treatments, but there were not significant differences between the response rates for the other treatments. Among our mixed-mode treatments, the sequential web-mail treatment had the highest percentage of response by web and the lowest cost per response. When focusing on the subgroups that we expected to be the least—and the most—comfortable with the internet, we found that the paper-only treatment generally performed worse than the others, even among subgroups expected not to be comfortable with the internet. We generally did not find significant differences in the effect of response mode treatment on the response rate or percentage of response by web between the subgroups who were the most and least comfortable with the internet. In terms of the consistency of response mode choice over time, our results suggest that some people respond consistently—but also that response mode preferences are weak enough that they can be influenced by the way in which the modes are offered. We ultimately recommend using a sequential web-mail design to minimize costs while still providing people who cannot or will not respond by web with another response mode option. We also find evidence that there may be a growing lack of interest in responding by paper; more research is needed in this area.
  PubDate: Tue, 30 Apr 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae021
  Issue No: Vol. 12, No. 3 (2024)
Effects of a Web–Mail Mode on Response Rates and Responses to a Care
Experience Survey: Results of a Randomized Experiment
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 624 - 650
  Abstract: AbstractPatient experience surveys are vital to evaluating healthcare provider performance. However, declining response rates over time and questions about whether responses reflect the perspectives of all patients under care have raised concerns. One proposed approach to address these concerns is web-based survey administration, a mode that has not been studied in the hospice setting. We tested a sequential web–mail mode for administering a care experience survey in this unique setting, where family caregivers respond after the patient dies. Sampled caregivers of 15,515 patients who died March–August 2021 while receiving care from 56 hospices across the US were randomized to one of four survey modes: mail-only, telephone-only, mail–telephone (mail with telephone follow-up), or web–mail (email invitation to a web survey with mail follow-up). Email addresses were available for 31.3 percent of sampled eligible caregivers.Relative to mail-only (estimated response rate = 35.1 percent), response rates were significantly higher for web–mail (39.7 percent) and mail–telephone (45.3 percent) and significantly lower for telephone-only (31.5 percent). The web–mail response rate was similar to the mail-only response rate among caregivers without email addresses (35.2 versus 34.3 percent), but substantially higher among caregivers with email addresses (49.6 versus 36.7 percent). Web–mail and mail-only respondents reported similar experiences for 26 of 27 evaluative items. Among eligible sampled caregivers, several patient/caregiver characteristics differed by caregivers’ email address availability, but web–mail and mail-only respondents did not differ on any characteristic. A web–mail mode is feasible for surveying bereaved caregivers about care experiences, producing substantially higher response rates than single-mode approaches, with increasing benefits for hospices with higher proportions of caregivers with email addresses. Findings may be applicable to surveys of other sensitive topics and to populations that prefer asynchronous survey modes.
  PubDate: Tue, 23 Apr 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae013
  Issue No: Vol. 12, No. 3 (2024)
Text Messages to Facilitate the Transition to Web-First Sequential
Mixed-Mode Designs in Longitudinal Surveys
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 651 - 673
  Abstract: AbstractThis article is concerned with the transition of a longitudinal survey from a single-mode design to a web-first mixed-mode design and the role that text messages to sample members can play in smoothing that transition. We present the results of an experiment that investigates the effects of augmenting the contact strategy of letters and emails with text messages, inviting the sample members to complete a web questionnaire and reminding them of the invite. The experiment was conducted in a subsample of Understanding Society, a household panel survey in the United Kingdom, in the wave that transitioned from a CAPI-only design to a sequential design combining web and CATI. In the experiment, a quarter of the sample received letters and emails, while the rest received between one and three text messages with a personalized link to the questionnaire. We examine the effect of the text messages on response rates, both at the web phase of a sequential design and at the end of the fieldwork after a CATI follow-up phase, and explore various mechanisms that might drive the increase in response rates. We also look at the effects on the device used to complete the survey and field efforts needed at the CATI stage. The findings indicate that text messages did not help to significantly increase response rates overall, although some subgroups benefited from them, such as panel members who had not provided an email or postal address before. Likewise, the text messages increased web completion among younger panel members and those with an irregular response pattern. We only found a slight and nonsignificant effect on smartphone use and no effect on the web household response rate, a proxy for fieldwork efforts.
  PubDate: Tue, 19 Mar 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae003
  Issue No: Vol. 12, No. 3 (2024)
When to Text' How the Timing of Text Message Contacts in Mixed-Mode
Surveys Impacts Response
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 674 - 696
  Abstract: AbstractUsing multiple modes of contact has been found to increase survey participation over a single contact mode. Text messaging has emerged as a new mode to contact survey participants in mixed-mode survey designs, especially for surveys that include web and/or phone data collection. However, it is unclear how to best combine text messages with mailings and other outreach contacts to improve response rates and data quality. To explore the effectiveness of using text messaging as a contact mode, we conducted a full factorial experiment that varies the sequencing of text messages with mailing contacts (early versus late reminder) and the time text messages were sent (morning versus afternoon). The experiment was implemented in a follow-up wave of a mixed-mode nationally representative longitudinal survey with two sample groups (Cooperative versus Other Respondents). For Cooperative Respondents, text reminders seemed to be effective at increasing completion rates, with the early text reminder being somewhat more effective than the late text reminder, at least early in the field period. For Other Respondents, text invitations were effective at improving the completion rate, but effects diminished quickly once the invitation letter was sent. Additionally, the early text reminder appears to be more effective than the late text reminder at increasing completion rates for Other Respondents. The sequencing of text messages did not affect data quality across sample groups or substantially impact nonresponse. The time of day the text messages were sent did not affect any of the outcome measures examined.
  PubDate: Mon, 29 Apr 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae014
  Issue No: Vol. 12, No. 3 (2024)
Supplementing a Paper Questionnaire with Web and Two-Way Short Message
Service (SMS) Surveys
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 697 - 711
  Abstract: AbstractWhen deciding which modes to offer, researchers consider cost, known respondent contact information, and potential mode effects. For a short survey on employment, we evaluate the effect of adding one of two new electronic data collection modes to a mailed questionnaire. We sent a survey to principals who previously responded to the National Center for Education Statistics’ (NCES) National Teacher and Principal Survey (NTPS) asking about their current job status. This questionnaire, known as the Principal Follow-up Survey (PFS), has typically been administered as a short paper form that is mailed to NTPS respondents. In 2022, the PFS introduced two new modes of completion, and principals were randomly assigned to receive: (i) a paper form only; (ii) a paper form, as well as emails with a direct link to complete a web survey; or (iii) a paper form, as well as invitations by text message to complete an automated two-way short message service text survey by responding to texted “yes/no” questions. This article compares overall response rates and time-to-response by mode to determine respondent preferences for completing short surveys. Adding either electronic mode significantly increased response rates and decreased the number of days in which completed surveys were received, compared to offering only a paper questionnaire. Although email and text messages are both forms of electronic communication that may be accessible on a smartphone, the added text message survey resulted in higher response rates than the added web survey. This suggests that respondents interact differently with emails and text messages they receive and that offering an option to complete a survey by text message can increase the speed and efficiency of data collection for short surveys.
  PubDate: Mon, 25 Mar 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae006
  Issue No: Vol. 12, No. 3 (2024)
Improving the Efficiency of Outbound CATI As a Nonresponse Follow-Up Mode
in Address-Based Samples: A Quasi-Experimental Evaluation of a Dynamic
Adaptive Design
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 712 - 740
  Abstract: AbstractThis article evaluates the use of dynamic adaptive design methods to target outbound computer-assisted telephone interviewing (CATI) in the California Health Interview Survey (CHIS). CHIS is a large-scale, annual study that uses an address-based sample (ABS) with push-to-Web mailings, followed by outbound CATI follow-up for addresses with appended phone numbers. CHIS 2022 implemented a dynamic adaptive design in which predictive models were used to end dialing early for some cases. For addresses that received outbound CATI follow-up, dialing was paused after three calls. A response propensity (RP) model was applied to predict the probability that the address would respond to continued dialing, based on the outcomes of the first three calls. Low-RP addresses were permanently retired with no additional dialing, while the rest continued through six or more attempts. We use a difference-in-difference design to evaluate the effect of the adaptive design on calling effort, completion rates, and the demographic composition of respondents. We find that the adaptive design reduced the mean number of calls per sampled unit by about 14 percent (relative to a modeled no-adaptive-design counterfactual) with a minimal reduction in the completion rate and no strong evidence of changes in the prevalence of target demographics. This suggests that RP modeling can meaningfully distinguish between ABS sample units for which additional dialing is and is not productive, helping to control outbound dialing costs without compromising sample representativeness.
  PubDate: Tue, 05 Mar 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae005
  Issue No: Vol. 12, No. 3 (2024)
The Efficacy of Propensity Score Matching for Separating Selection and
Measurement Effects Across Different Survey Modes
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 764 - 789
  Abstract: AbstractEffective evaluation of data quality between data collected in different modes is complicated by the confounding of selection and measurement effects. This study evaluates the utility of propensity score matching (PSM) as a method that has been proposed as a means of removing selection effects across surveys conducted in different modes. Our results show large differences in estimates for the same variables between parallel face-to-face and online surveys, even after matching on standard demographic variables. Moreover, discrepancies in estimates are still present after matching between surveys conducted in the same (online) mode, where differences in measurement properties can be ruled out a priori. Our findings suggest that PSM has substantial limitations as a method for separating measurement and selection differences across modes and should be used only with caution.
  PubDate: Wed, 17 Apr 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae017
  Issue No: Vol. 12, No. 3 (2024)
A Cost–Benefit Analysis of Reinterview Designs for Estimating and
Adjusting Mode Measurement Effects: A Case Study for the Dutch Health
Survey and Labour Force Survey
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 790 - 813
  Abstract: AbstractReinterview designs are a potential tool to estimate and adjust for mode measurement effects, that is, relative differences in mode-specific measurement error bias. In 2011, a reinterview design was successfully applied to the Dutch Crime Victimization Survey, which led to a redesign of the survey. Reinterview designs may, however, be very costly, especially when face to face is included as a survey mode. The crucial question is whether benefits outweigh costs, that is, whether the potential increase in the accuracy of survey statistics is worth the investment. The answer to this question depends heavily on the purpose of the reinterview, that is, assessment versus adjustment, the size of the measurement effects, and the relative cost of the modes. Reinterview designs also make a number of assumptions that will not hold for every setting. In this article, we perform a cost–benefit analysis for two surveys, the Dutch Health Survey and the Dutch Labour Force Survey, and discuss the utility and validity of reinterviews. We conclude that a reinterview may not be useful due to relatively small measurement differences for the Labour Force Survey, whereas it may be useful for the Health Survey.
  PubDate: Thu, 28 Mar 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae011
  Issue No: Vol. 12, No. 3 (2024)
Three Approaches to Improve Inferences Based on Survey Data Collected with
Mixed-mode Designs
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 814 - 839
  Abstract: AbstractMixed-mode designs have become increasingly common in survey data collection. Although different modes often have different measurement properties, the standard practice is to treat mixed-mode data as if they had been collected with a single mode, neglecting the potential impact of mode effects. To account for potential mode effects when making inferences for mixed-mode samples, we propose (i) a Testimator approach, (ii) a Bayesian approach, and (iii) a model averaging approach. In the Testimator approach, we test whether the means and the variances of mixed-mode samples are the same. If the means are the same, we take the average of mode-specific estimates. If the means are different, we take the average when we have no prior information about preferred modes and take the smaller (or larger) estimate when we have prior information about preferred modes (e.g., a smaller estimate is better). In the Bayesian approach, we assume some prior information. We use a data-driven method to determine whether there are mode effects. If there are no mode effects, we draw inferences using a common mean model. If there are mode effects, we draw inferences using the data collected with the mode that produces smaller estimates. In the model averaging approach, we combine estimates of different models (characterized by whether assume same means and variances across modes) using marginal posteriors as weights. We evaluate the approaches in simulation studies and find that they achieve robust inferences compared to the standard approach. We apply the methods to the Arab Barometer study, which employs a randomized mixed-mode design.
  PubDate: Mon, 29 Apr 2024 00:00:00 GMT
  DOI: 10.1093/jssam/smae012
  Issue No: Vol. 12, No. 3 (2024)
Incorporating Adaptive Survey Design in a Two-Stage National Web or Mail
Mixed-Mode Survey: An Experiment in the American Family Health Study
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 578 - 592
  Abstract: AbstractThis article presents the results of an adaptive design experiment in the recruitment of households and individuals for a two-stage national probability web or mail mixed-mode survey, the American Family Health Study (AFHS). In the screening stage, we based the adaptive design’s subgroup differentiation on Esri Tapestry segmentation. We used tailored invitation materials for a subsample where a high proportion of the population was Hispanic and added a paper questionnaire to the initial mailing for a subsample with rural and older families. In the main-survey stage, the adaptive design targeted the households where a member other than the screening respondent was selected for the survey. The adaptations included emailing and/or texting, an additional prepaid incentive, and seeking screening respondents’ help to remind the selected individuals. The main research questions are (i) whether the adaptive design improved survey production outcomes and (ii) whether combining adaptive design and postsurvey weighting adjustments improved survey estimates compared to performing postsurvey adjustments alone. Unfortunately, the adaptive designs did not improve the survey production outcomes. We found that the weighted AFHS estimates closely resemble those of a benchmark national face-to-face survey, the National Survey of Family Growth, although the adaptive design did not additionally change survey estimates beyond the weighting adjustments. Nonetheless, our experiment yields useful insights about the implementation of adaptive design in a self-administered mail-recruit web or mail survey. We were able to identify subgroups with potentially lower response rates and distinctive characteristics, but it was challenging to develop effective protocol adaptations for these subgroups under the constraints of the two primary survey modes and the operational budget of the AFHS. In addition, for self-administered within-household selection, it was difficult to obtain contact information from, reach, and recruit selected household members that did not respond to the screening interview.
  PubDate: Tue, 12 Sep 2023 00:00:00 GMT
  DOI: 10.1093/jssam/smad035
  Issue No: Vol. 12, No. 3 (2023)
Optimizing Data Collection Interventions to Balance Cost and Quality in a
Sequential Multimode Survey
- Free pre-print version: Loading...
  Rate this result: What is this?
  
  Please help us test our new pre-print finding feature by giving the pre-print link a rating.
  A 5 star rating indicates the linked pre-print has the exact same content as the published article.
  
  Pages: 741 - 763
  Abstract: AbstractHigh-quality survey data collection is getting more expensive to conduct because of decreasing response rates and rising data collection costs. Responsive and adaptive designs have emerged as a framework for targeting and reallocating resources during the data collection period to improve survey data collection efficiency. Here, we report on the implementation and evaluation of a responsive design experiment in the National Survey of College Graduates that optimizes the cost-quality tradeoff by minimizing a function of data collection costs and the root mean squared error of a key survey measure, self-reported salary. We used a Bayesian framework to incorporate prior information and generate predictions of estimated response propensity, self-reported salary, and data collection costs for use in our optimization rule. At three points during the data collection process, we implement the optimization rule and identify cases for which reduced effort would have minimal effect on the mean squared error (RMSE) of mean self-reported salary while allowing us to reduce data collection costs. We find that this optimization process allowed us to reduce data collection costs by nearly 10 percent, without a statistically or practically significant increase in the RMSE of mean salary or a decrease in the unweighted response rate. This experiment demonstrates the potential for these types of designs to more effectively target data collection resources to reach survey quality goals.
  PubDate: Sat, 08 Apr 2023 00:00:00 GMT
  DOI: 10.1093/jssam/smad007
  Issue No: Vol. 12, No. 3 (2023)

HOME > Browse the 73 Subjects covered by JournalTOCs
Subject	Total Journals