The Use of Experimental Methods by IS Scholars: An Illustrated Typology

Marta Ballatore (1) , Lise Arena (1) , Agnès Festré (1)

1 GREDEG - Groupe de Recherche en Droit, Economie et Gestion

This article aims at making an updated typology of recent experimental studies in the IS literature on the period 1999-2019. Based on a full-text search within the Association for Information Systems (AIS) "basket" of eight top IS journals (EJIS, ISR, JAIS, ISJ, JIT, JMIS, JSIS and MISQ), this research gathered 392 articles and highlights the use of 4 different types of experiments in IS, mainly: laboratory experiments , field experiments, online experiments (scenario simulation game-based; brainstorming-based.. .) and natural experiments. Each category is discussed through the perspective of its degree of control, and technological realism. Results show the significant predominance of laboratory experiments over field and natural experiments on the period. This, in turn, stresses the preferred tendency followed by IS scholars to perceive experimental methods as a way to control the source of variations of variables under study. In addition, this paper provides a better understanding of the context of use of a specific experimental method. Overall, it is shown that laboratory experiments (including scenario-based lab experiments) are mainly used, in a deterministic manner, to assess or test the impact of an IS on human decision-making or behaviour. By contrast, artificial simulations experiments are more appropriate to study emergent phenomena and to make predictions, often providing key insights about quality and effectiveness of IS. ⇤ The authors are grateful to anonymous referees of the scientific committee of the AIM 2020 for their insights comments and suggestions.

Mots clés: Methodology; IS; Meta-research; Experimental method; Behavioural Science; Experimental Economics

Domaines: Gestion et management; Economies et finances

Marta Ballatore, Lise Arena, Agnès Festré. The Use of Experimental Methods by IS Scholars: An Illustrated Typology. 25ème Conférence de l'Association Information et Management (AIM), Jun 2020, Marrakech, Morocco. ⟨halshs-02866756⟩