Lazy Gatekeepers: A Large-Scale Study on SPF Configuration in the Wild

This repository belongs to the publication:

Lazy Gatekeepers: A Large-Scale Study on SPF Configuration in the Wild. Stefan Czybik, Micha Horlboge and Konrad Rieck. Proc. of the 23rd ACM Internet Measurement Conference (IMC), 2023.

@inproceedings{CzyHorRie23,
        title={Lazy Gatekeepers: A Large-Scale Study on SPF Configuration in the Wild.},
        author={Stefan Czybik and Micha Horlboge and Konrad Rieck},
        year={2023},
        booktitle={Proc. of the 23rd ACM Internet Measurement Conference ({IMC})},
  }

Background

This repository contains a scanner to scan and parse SPF records from domains within a database.

Getting Started

Make sure you have a python environment and docker running.

git clone git@gitlab.com:stcz/spf-scanner.git
cd spf-scanner

Setup your python environment:

pip install -r requirements.txt

To map IP addresses to ASNs download latest pyasn data:

python pyasn_util_download.py --latestv4
python pyasn_util_convert.py --single insert_downloaded_file_here asn_db.dat

Edit .env and change the passwords. Further edit docker/docker-compose.yml and change the passwords too.

Setup mariaDB in a docker environment

cd docker
docker compose up -d
cd ..

Now login to http://localhost:8086 and add the domains you want to scan to the table domains.

Then you can start the scan:

python scan_domains.py