Pretending not to know

Matan Mazor 🙈, Chaz Firestone 🙉 & Ian Phillips 🙊

A series of experiments looking at people's ability to pretend they don't know something.

Data

Raw data from all four experiments is available in the "experiments" subdirectory.

Battleship:

Under experiments/Battleship2/data:

• debrief.csv records participants' responses to the debriefing question.
• under experiments/Battleship2/data/batch1:
• prolific_export_batch1.csv records demographic data
• jatos_results_batch1.csv records experimental data.
• jatos_results_batch1_readme.txt includes an explanation of each column in jatos_results_batch1.csv
• bonus.csv records participants' bonuses
• Click_log and Click_log_with_boards are preprocessed data from the experiments themselves.

Hangman:

Under experiments/Hangman2/data:

• debrief.csv records participants' responses to the debriefing question.
• prolific_export1.csv to rolific_export6.csv records demographic data
• jatos_results_batch1.csv and jatos_results_batch2.csv record experimental data from the two batches of participants.
• jatos_results_batch1_readme.txt includes an explanation of each column in the above files
• E4.click_log and E4.click_log_with_boards and E4.judge_df are preprocessed data from the experiments themselves, which can be generated with the analysis code.

Analysis Scripts

All pre-registered analyses, exploratory analyses, and the paper itself in a data-to-paper Rmarkdown format are available in the 'docs' subdirectory.

The outputs of the scripts are available in gitbook format (both preregistered and exploratory analyses).

To reproduce the paper, follow these steps:

1. The scripts use the R groundhog package for version control. This package needs to be installed. The package also requires that the scripts are run on R version 4.2.2 ("Innocent and Trusting"). It should be possible to run it on newer versions, but then the calls to groundhog.library would need to specify a more recent date.

2. The scripts should be run in the order 'preregisteredMethodsAndResults.Rmd','exploratoryResults.Rmd', and only then 'paper.Rmd'. The reason is that the first two scripts save their output as RData files, which the third scripts reads. All scripts can be found in the 'docs' folder.

Experiment demos

You can try demos of the two experiments, as experienced by our participants.

Pre-registration and pre-registration time-locking 🕝🔒

OSF pre-registrations are available for Battleship and Hangman.

To ensure preregistration time-locking (in other words, that preregistration preceded data collection), we employed randomization-based preregistration. We used the SHA256 cryptographic hash function to translate our preregistered protocol folder (including the pre-registration document) to a string of 256 bits. These bits were then combined with the unique identifiers of single subjects, and the resulting string was used as seed for initializing the Mersenne Twister pseudorandom number generator prior to determining all random aspects of the experiment, including the order of game boards (grid configurations or target words) presented to participants, and their assignment of boards to experimental conditions. This way, experimental randomization was causally dependent on, and therefore could not have been determined prior to, the specific contents of our preregistration document (Mazor, Mazor & Mukamel, 2019).

Exp. 1: Battleship

protocol folder

protocol sum: 60c270410375e8a192468fc1a0e9c93da60d5e203eb2760b621a8631a26f4c5c

relevant pre-registration lines of code

Exp. 2: Hangman

protocol folder

protocol sum: c4929c7fe33df1b7b52f15c789d98eab30a9cee09a8121807a3c59e28e7430a4

relevant pre-registration lines of code

⚠️ Notice that the due to an error in the experimental code, only the second batch of Hangman participants was effectively time-locked. See here.