ICA 2027 Preconference
From Outputs to Evidence: Validating GenAI-Enabled Methods in Communication Research
Date: June 2, 2027
Location: University of Strathclyde, Glasgow, United Kingdom

Organized by
Dr. Winson Peng
Michigan State University, USA		 Dr. Dayei Oh
University of Strathclyde, UK
Dr. Yingdan Lu
Northwestern University, USA		 Dr. Guang Yang
Shenzhen University, China

Overview

Generative AI (GenAI) is rapidly becoming part of the research infrastructure of communication research. Researchers are using large language models (LLMs) to code and interpret communication content, augment survey research, generate and personalize experimental stimuli, simulate communication processes, actors, and environments, and develop new forms of data collection and analysis. These developments offer substantial opportunities for methodological innovation. At the same time, they raise a more fundamental question:

When can outputs produced or assisted by GenAI be treated as valid scientific evidence about human communication?

Recent methodological work has highlighted the importance of this question. Dell and Rambachan (2026), for example, argue that AI is transforming the measurement process by making alternative measures increasingly easy to construct. This shift moves attention from the availability of measures to the credibility of the criteria and validation procedures used to select among them. Salaudeen et al. (2025) similarly emphasize that evaluation should be considered in relation to the claims researchers seek to make: evidence that supports a narrowly defined performance claim may not be sufficient to support broader claims about an underlying construct.

These concerns are particularly consequential for communication research. High agreement between an AI system and human coders does not necessarily demonstrate that the system captures the intended theoretical construct. Realistic AI-generated survey responses may not establish measurement equivalence or population validity. Personalized experimental stimuli may introduce new questions about treatment validity, experimental control, and reproducibility. Generative agents may reproduce recognizable patterns of communication without reproducing the psychological, interpersonal, or social mechanisms that generate those patterns among humans.

The preconference will bring together communication scholars, methodologists, computational researchers, and AI ethicists to examine how validation should be conceptualized, implemented, and reported as GenAI becomes increasingly integrated into communication research. The goal is not to establish a single universal validation standard. Rather, the preconference seeks to clarify what forms of validation evidence are needed for different research purposes, methods, populations, contexts, and levels of inference.

Topics of Interest

We invite conceptual, methodological, and empirical submissions addressing issues including, but not limited to:

  • GenAI-enabled content analysis and computational measurement, including construct definition, human-AI agreement, benchmark selection, measurement error, and validation of AI-generated variables;
  • GenAI-enabled surveys and interviewing, including AI-generated probes, conversational surveys, synthetic responses, measurement equivalence, response processes, and population validity;
  • GenAI-enabled experiments, including generated or personalized stimuli, treatment validity, experimental control, robustness, transparency, and reproducibility;
  • GenAI-enabled social simulation and generative agents, including behavioral and generative validity, mechanism validation, correspondence with human behavior, and appropriate limits of inference;
  • Validation frameworks for GenAI-enabled research, including construct, criterion, content, external, and inferential validity;
  • Robustness and sensitivity of GenAI-enabled methods across models, prompts, training procedures, samples, populations, languages, and social contexts;
  • Human benchmarks and validation data, including the role and limitations of expert judgments, human annotations, validation samples, and conventional measurement instruments;
  • Relationships among measurement, evidence, and scientific claims, particularly the conditions under which AI-generated outputs can support broader theoretical or empirical conclusions;
  • Transparency, reproducibility, and reporting standards for GenAI-enabled communication research;
  • Other conceptual or methodological issues concerning the use and validation of GenAI throughout the communication research process.

We particularly welcome submissions that move beyond demonstrating that a GenAI method works and instead examine what its outputs allow researchers to claim, what evidence is needed to support those claims, and where the boundaries of those claims should be drawn.

Submission Guideline

Participants should submit an extended abstract of no more than 500 words. The 500-word limit excludes references, tables, and figures.

Extended abstracts should clearly describe:
- the research problem or methodological issue;
- the approach or argument advanced; and
- its relevance to the validation of GenAI-enabled methods in communication research.

The submission must also include the following information for each author:
- Full name
- Institutional affiliation
- Email address
- Indication of the presenting author(s)

Author information does NOT count toward the 500-word limit.

Please email your submission to: ica2027aiprecon@gmail.com
File format: PDF
File name: FirstAuthorFirstName_FirstAuthorLastName_2027.pdf

Completed studies and well-developed works in progress are welcome, as are conceptual and methodological contributions.

Submissions will be reviewed on the basis of their relevance to the preconference theme, conceptual or methodological contribution, clarity of argument, and potential to advance understanding of the reliable and valid use of GenAI in communication research.

Important Dates

Submission deadline: January 15, 2027 (U.S. Eastern Time)
Notification of acceptance: February 14, 2027 (U.S. Eastern Time)

Registration

Registration fee: USD 50 per participant.
The registration fee will help cover preconference expenses, including refreshments and, subject to available funding, lunch.