Skip to content

UCSB-AMPLab/digitization-toolkit-software

Repository files navigation

Digitization Toolkit

An open-source, modular digitization toolkit designed for low-cost, high-quality scanning using Raspberry Pi Compute Module 4 and 64MP ArduCam cameras. Built for use in low-resource environments and community archives.

Goals

  • Empower small institutions and communities to digitize archival materials.
  • Provide an easy-to-use web interface for image capture, metadata entry, and export.
  • Ensure long-term reproducibility through open standards (BagIt, CSV, TIFF).
  • Prioritize modularity and hardware independence for future-proofing.

Tech Stack

  • Hardware: Raspberry Pi CM4 + CM4IO, Raspberry Pi 5 (dual camera embedded), 2x 64MP ArduCam autofocus cameras, 2 x Cannon EOS Rebel T7 (GPhoto compatible cameras)
  • Backend: Python, FastAPI
  • Frontend: Svelte
  • Database: PostgreSQL

Status

Project currently in development. Kick-off: September 2025 Alpha prototype planned for deployment at SBMAL in June 2025.


Quick Start

Development (any machine, no cameras)

# One-command startup (all services in Docker)
./scripts/start-dev.sh

Or manually:

docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile with-backend up

Production (Raspberry Pi with cameras)

# Build the frontend image first (requires internet — do this at home/office):
docker compose build

# Then take it offline and run:
./scripts/start.sh

Or manually:

# Start database and frontend
docker compose up -d

# Start backend with pixi
cd backend && pixi run dev

Access (production): http://localhost:3000 (frontend) | http://localhost:8000/docs (API) Access (dev): http://localhost:5173 (frontend, Vite) | http://localhost:8000/docs (API)

Note: The native backend is required for camera access due to Raspberry Pi-specific libraries (libcamera, picamera2).

📦 Backend Dependency Management

The backend uses pixi for dependency management:

# First time setup
cd backend
pixi install
pixi run setup-camera-link  # Raspberry Pi only

# Start backend
pixi run dev

Production Distribution (SD Card Imaging)

The toolkit is designed to ship as a pre-flashed SD card. The end user only needs to insert the card, power on the Pi, and the application starts automatically — no internet connection required.

Building the golden SD card (requires internet, done once)

# 1. Clone the repository onto a fresh Raspberry Pi OS installation
git clone --recurse-submodules https://github.com/UCSB-AMPLab/digitization-toolkit.git ~/dtk
cd ~/dtk

# 2. Run the one-time provisioning script (internet required here)
#    Builds Docker images, installs pixi env, applies DB migrations
sudo ./scripts/setup.sh

# 3. Install and enable the systemd service (auto-start on boot)
sudo ./scripts/install-service.sh
sudo systemctl start dtk

# 4. Verify the app is running
curl http://localhost:8000/health   # → {"status":"ok"}
curl -I http://localhost:3000       # → HTTP 200

Creating an SD card image

# On the Pi — shut down cleanly
sudo systemctl stop dtk
sudo poweroff

# On a host machine with the SD card inserted (replace sdX with your device)
sudo dd if=/dev/sdX of=dtk-$(date +%Y%m%d).img bs=4M status=progress
# Compress for storage/transfer
xz -T0 dtk-$(date +%Y%m%d).img

Flashing for end users

Use Raspberry Pi Imager (or dd) to write the .img to a new SD card.
The application starts automatically on boot — no configuration needed.

What's self-contained after provisioning

Component Stored at Requires internet after setup?
Frontend (SvelteKit) Docker image in /var/lib/docker/ No
PostgreSQL Docker image in /var/lib/docker/ No
Backend Python env backend/.pixi/ No
camera libraries /usr/lib/python3/dist-packages/ No (system OS)
Application data /var/lib/dtk/ No

Setup

This repository uses Git submodules for the frontend and backend code.
When cloning, make sure to fetch submodules as well:

# Clone with submodules
git clone --recurse-submodules git@github.com:UCSB-AMPLab/digitization-toolkit.git

If you already cloned without --recurse-submodules, you can initialize and update submodules manually:

git submodule update --init --recursive

To pull the latest changes for submodules after updates:

git submodule update --remote --merge

Development Documentation

See the wiki for detailed developer guides and API references.

License

MIT

About

Core repository for the Digitization Toolkit Software

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors