Zotero-FormBD

image:[License,link="https://github.com/hyperpolymath/palimpsest-license"]

Zotero-FormBD

A narrative and provenance based FormBD application for the reference manager Zotero. Also works on cloud-drives.

Quick Start

Choose your platform:

Linux / macOS - Linux and macOS
Windows - Windows 10/11

Linux / macOS

Path A: I Already Use Zotero

You have Zotero with a library. You want FormBD’s provenance tracking and cloud-safe storage.

Time: ~5 minutes

# 1. Install Julia (if you don't have it)
curl -fsSL https://install.julialang.org | sh
# Then restart your terminal, or run: source ~/.bashrc

# 2. Run the installer (shows what it will do first)
curl -sSL https://raw.githubusercontent.com/hyperpolymath/zotero-FormBD/main/install.sh | bash

# 3. If the plan looks good, actually install
curl -sSL https://raw.githubusercontent.com/hyperpolymath/zotero-FormBD/main/install.sh | bash -s -- --apply

# 4. Start the server
FormBD-server

# 5. Test it
curl http://localhost:8080/users/local/items?limit=3

What just happened:

Your Zotero library was copied (not moved!) to ~/.FormBD/zotero/
Each item now has provenance tracking ("migrated from Zotero")
You have a Zotero-compatible API server on localhost:8080
Your original Zotero is untouched and still works normally

Keep them in sync:

# With Zotero running, sync any changes to FormBD
FormBD-sync

Path B: I’m New to Reference Management

You’ve been told this is a better way 'to do' Zotero. You don’t have Zotero. Let’s get you set up from scratch.

Time: ~10 minutes

Step 1: Install Zotero (the UI)

FormBD uses Zotero’s excellent UI. Install Zotero first:

# Fedora/RHEL
sudo dnf install zotero

# Ubuntu/Debian
sudo apt install zotero

# Or download from: https://www.zotero.org/download/

Step 2: Install Julia (the engine)

curl -fsSL https://install.julialang.org | sh
# Restart your terminal after this

Step 3: Open Zotero Once

Open Zotero from your applications menu. This creates your library. You can close it after.

Step 4: Install FormBD

# Preview the installation
curl -sSL https://raw.githubusercontent.com/hyperpolymath/zotero-FormBD/main/install.sh | bash

# If it looks good, install
curl -sSL https://raw.githubusercontent.com/hyperpolymath/zotero-FormBD/main/install.sh | bash -s -- --apply

Step 5: Daily Workflow

# 1. Use Zotero normally to add papers, books, etc.
#    (Browser connector works, drag-and-drop works, everything works)

# 2. When you want your changes in FormBD, run:
FormBD-sync

# 3. To query your library programmatically:
FormBD-server &
curl http://localhost:8080/users/local/items

Windows

Prerequisites

Install Julia: Download from https://julialang.org/downloads/ and run the installer
Install Git: Download from https://git-scm.com/download/win
Install Zotero: Download from https://www.zotero.org/download/
Open Zotero once to create your library, then close it

Installation

Open PowerShell and run:

# 1. Download the installer
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/hyperpolymath/zotero-FormBD/main/install.ps1" -OutFile "install.ps1"

# 2. Preview what will happen
.\install.ps1

# 3. If the plan looks good, actually install
.\install.ps1 -Apply

# 4. Restart your terminal (for PATH changes)

# 5. Start the server
FormBD-server

# 6. Test it
Invoke-RestMethod http://localhost:8080/users/local/items?limit=3

Daily Workflow

# 1. Use Zotero normally to add papers, books, etc.

# 2. When you want your changes in FormBD, run:
FormBD-sync

# 3. To query your library:
FormBD-server
# Then in another terminal:
Invoke-RestMethod http://localhost:8080/users/local/items

Uninstall

# Option 1: Use the uninstall script
~\.FormBD\uninstall.ps1

# Option 2: Re-run installer with -Uninstall
.\install.ps1 -Uninstall

What Do I Get?

Feature	Regular Zotero	Zotero + FormBD
Add papers from browser	✓	✓ (use Zotero, then sync)
Organize into collections	✓	✓
Cloud sync (Dropbox, etc.)	⚠️ Corruption risk	✓ Safe (append-only)
"Who added this?"	❌ Not tracked	✓ Required
"Why was this added?"	❌ Optional note	✓ Required rationale
Undo any change	❌ Hope you have backups	✓ Full history
Query via API	❌ Need plugins	✓ REST API included
Evidence quality scores	❌	✓ PROMPT framework
Publisher quality tracking	❌	✓ Ownership & bias ratings
Funding transparency	❌	✓ "Who paid for this?"
Library blindspot detection	❌	✓ Like Ground News for papers

Commands Reference

After installation, you have seven commands:

FormBD-server [--port 8080] [--host 127.0.0.1]
    Start the Zotero-compatible REST API server.
    Your library is available at http://localhost:8080/users/local/items

FormBD-sync [--dry-run] [--verbose]
    Sync changes from running Zotero to FormBD.
    Requires Zotero to be open (uses local API on port 23119).

FormBD-migrate --apply
    Re-run full migration from Zotero SQLite.
    Use this if sync gets out of date or you want a fresh start.

FormBD-score --item KEY [--provenance N] [--methodology N] ...
    Set PROMPT evidence quality scores for an item (v0.2.0).
    Each dimension is 0-100. Use --all N for quick scoring.
    Use --list-unscored to find items needing evaluation.

FormBD-doi --status KEY
    Check DOI status of an item (v0.3.0).
    Shows if item is canonical (immutable) or a play-variant.

FormBD-doi --create-variant KEY [--name NAME]
    Create an editable play-variant of a canonical DOI item.
    Notes and annotations can only be attached to variants.

FormBD-doi --list-doi
    List all canonical DOI items in your library.

FormBD-doi --list-variants
    List all play-variants you've created.

API Examples

# List all items
curl http://localhost:8080/users/local/items

# Get a specific item
curl http://localhost:8080/users/local/items/ABC12345

# List collections
curl http://localhost:8080/users/local/collections

# Get items in a collection
curl http://localhost:8080/users/local/collections/XYZ98765/items

# Search for items
curl "http://localhost:8080/users/local/items?q=climate+change"

# Filter by item type
curl "http://localhost:8080/users/local/items?itemType=journalArticle"

# Paginate
curl "http://localhost:8080/users/local/items?limit=50&start=100"

# PROMPT Scoring (v0.2.0)
# Set evidence quality scores for an item
curl -X PUT "http://localhost:8080/users/local/items/ABC12345/prompt-scores" \
    -H "Content-Type: application/json" \
    -d '{"provenance": 90, "methodology": 85, "publication": 95}'

# Get PROMPT scores for an item
curl "http://localhost:8080/users/local/items/ABC12345/prompt-scores"

# Filter by minimum quality score
curl "http://localhost:8080/users/local/items?minScore=80"

# Get only items that have been scored
curl "http://localhost:8080/users/local/items?hasScore=true"

# DOI Immutability (v0.3.0)
# Check if an item is a canonical DOI item
curl "http://localhost:8080/users/local/items/ABC12345/doi-status"

# Create a play-variant for annotation
curl -X POST "http://localhost:8080/users/local/items/ABC12345/create-variant" \
    -H "Content-Type: application/json" \
    -d '{"variantName": "My Notes"}'

# List only canonical DOI items
curl "http://localhost:8080/users/local/items?hasDOI=true"

# List only play-variants
curl "http://localhost:8080/users/local/items?isVariant=true"

# Publisher Registry (v0.4.0)
# List all registered publishers
curl "http://localhost:8080/users/local/publishers"

# Get a specific publisher
curl "http://localhost:8080/users/local/publishers/nature"

# Add a publisher
curl -X PUT "http://localhost:8080/users/local/publishers/nature" \
    -H "Content-Type: application/json" \
    -d '{"name": "Nature Publishing Group", "ownership": "commercial_large", "website": "https://www.nature.com"}'

# Score a publisher
curl -X PUT "http://localhost:8080/users/local/publishers/nature/scores" \
    -H "Content-Type: application/json" \
    -d '{"peer_review_rigor": 95, "transparency": 80, "accessibility": 60}'

# Funding Tracking (v0.4.0)
# Get funding info for an item
curl "http://localhost:8080/users/local/items/ABC12345/funding"

# Set funding info for an item
curl -X PUT "http://localhost:8080/users/local/items/ABC12345/funding" \
    -H "Content-Type: application/json" \
    -d '{"sources": [{"category": "government", "funder": "NSF", "grant_id": "123456"}]}'

# Multi-Scorer PROMPT (v0.4.0)
# Add another evaluator's scores (aggregates multiple opinions)
curl -X POST "http://localhost:8080/users/local/items/ABC12345/prompt-scores-multi" \
    -H "Content-Type: application/json" \
    -d '{"scorer_id": "reviewer2", "provenance": 85, "methodology": 90}'

# Get aggregated scores with consensus metrics
curl "http://localhost:8080/users/local/items/ABC12345/prompt-scores-multi"
# Returns: average, range, std_dev, consensus_level (high/medium/low)

# Blindspot Analysis (v0.4.0)
# Get full blindspot analysis
curl "http://localhost:8080/users/local/blindspots"
# Detects: funding_concentration, publisher_concentration, methodology_quality issues

Uninstall

# Remove FormBD completely (Zotero is NOT affected)
~/.FormBD/uninstall.sh

# Or
curl -sSL https://raw.githubusercontent.com/hyperpolymath/zotero-FormBD/main/install.sh | bash -s -- --uninstall

Overview

Zotero-FormBD reimagines bibliographic management for an era where source verification matters. Built on FQL-dt (FormBD Query Language with Dependent Types), it provides:

Provenance by construction - You cannot add an item without saying who added it and why
Evidence quality scores - PROMPT-style ratings (Provenance, Replicability, Objectivity, Methodology, Publication, Transparency)
Reversibility proofs - Every change can be undone, with mathematical proof
Cloud-safe storage - Append-only journal works on Dropbox, GDrive, OneDrive without corruption
Machine-checkable integrity - Types ARE proofs; the compiler verifies your data

Why Not Just SQLite?

Problem	SQLite	Zotero-FormBD
Cloud sync corruption	Common (concurrent writes)	Impossible (append-only)
"Who added this?"	Optional field, often empty	Required by type system
"Is this source reliable?"	Trust the user	PROMPT scores with bounds proofs
"Can I undo this?"	Maybe, check backups	Type carries reversibility proof
"Is data consistent?"	Runtime checks	Compile-time proofs

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Zotero (Browser Extension / Desktop App)                   │
│  You keep using Zotero normally                             │
├─────────────────────────────────────────────────────────────┤
│  Sync Bridge (FormBD-sync)                                  │
│  Pulls changes from Zotero's local API                      │
├─────────────────────────────────────────────────────────────┤
│  FormBD Journal (append-only)                               │
│  ~/.FormBD/zotero/journal.jsonl                             │
│  Content-addressed, hash-chained, provenance-tracked        │
├─────────────────────────────────────────────────────────────┤
│  REST API Server (FormBD-server)                            │
│  Zotero-compatible endpoints on localhost:8080              │
├─────────────────────────────────────────────────────────────┤
│  Cloud Storage (optional)                                   │
│  Just copy ~/.FormBD/ to Dropbox/GDrive/OneDrive            │
│  Append-only = no corruption from concurrent writes         │
└─────────────────────────────────────────────────────────────┘

Core Types (Lean 4)

-- Every item has provenance by construction
structure ZoteroItem where
  id : ValidUUID
  itemType : ItemType
  added_by : ActorId        -- Non-empty, required
  added_at : Timestamp
  rationale : Rationale     -- Non-empty, required

-- Evidence quality (PROMPT framework)
structure PromptScores where
  provenance : BoundedNat 0 100
  replicability : BoundedNat 0 100
  objective : BoundedNat 0 100
  methodology : BoundedNat 0 100
  publication : BoundedNat 0 100
  transparency : BoundedNat 0 100
  overall : BoundedNat 0 100
  overall_correct : overall.val = (provenance.val + replicability.val +
                    objective.val + methodology.val + publication.val +
                    transparency.val) / 6

DOI Immutability (v0.3.0)

When a document has a DOI, that DOI is a cryptographic promise: "this IS that document." FormBD treats DOIs as sources of truth for document identity.

The Rules

Operation	Canonical DOI Item	Play-Variant
Exists as	Immutable reference	Editable personal copy
Edits	❌ Blocked (creates variant instead)	✓ Allowed
Notes/Annotations	❌ Cannot attach	✓ Attach here
Tags	❌ Cannot modify	✓ Modify freely
Identity	You KNOW it’s that document	Clearly marked as "derived from DOI:xxx"

Why This Matters

A DOI identifies THE document as published. When you cite DOI:10.1000/xyz:

You’re citing the canonical, unmodified version
Not "your copy with annotations"
Not "a version you edited for clarity"

FormBD enforces this separation so your citations remain valid references to the actual source.

How It Works

Item with DOI:10.1000/xyz
├── [CANONICAL] The original, immutable
│   └── Cannot be edited, annotated, or tagged
│
└── [PLAY-VARIANT] Your personal copy
    ├── parent_doi: "10.1000/xyz"
    ├── Your annotations
    ├── Your notes
    └── Your tags

When you try to edit a DOI item, FormBD automatically: 1. Creates a play-variant linked to the original 2. Applies your changes to the variant 3. Leaves the canonical untouched

Troubleshooting

"Cannot connect to Zotero" when running FormBD-sync

Zotero must be running for sync to work. The sync bridge connects to Zotero’s local API on port 23119.

# Check if Zotero's API is available
curl http://localhost:23119/api/users/0/items?limit=1

"Julia not found"

Make sure Julia is installed and in your PATH:

# Install Julia
curl -fsSL https://install.julialang.org | sh

# Restart your terminal, then verify
julia --version

"Zotero database not found"

The installer looks for Zotero’s database in common locations. If yours is elsewhere:

# Find it manually
find ~ -name "zotero.sqlite" 2>/dev/null

# Then run migration manually
cd ~/.FormBD/repo/migration
julia --project=. bin/migrate.jl --from /path/to/your/zotero.sqlite --to ~/.FormBD/zotero --apply

Roadmap

v0.1.0: Core Types + Journal ✓

Lean 4 type definitions
Append-only journal format
SQLite migration tool
REST API server
Sync bridge

v0.2.0: Evidence Quality ✓

PROMPT scoring API (GET/PUT /items/:key/prompt-scores)
Quality-filtered queries (?minScore=80, ?hasScore=true)
FormBD-score CLI for evidence evaluation
Scores stored in journal with provenance

v0.3.0: DOI Immutability ✓

DOI items become canonical, immutable references
Edits to DOI items create "play-variants" (clearly marked forks)
Notes/annotations can ONLY attach to play-variants
DOI canonical versions remain pristine - you KNOW it’s that document
parentDOI field links variants back to canonical source
FormBD-doi CLI for DOI management
API endpoints: /doi-status, /create-variant
Query filters: ?hasDOI=true, ?isVariant=true

v0.4.0: Source Intelligence ✓

Publisher/Journal registry with ownership tracking (like Ground News)
Publisher quality scoring (peer review, transparency, accessibility)
Funding source tracking (who paid for this research?)
Multi-scorer PROMPT aggregation (multiple evaluators, consensus metrics)
Blindspot detection (funding concentration, publisher concentration)
FormBD-publisher CLI for publisher management
FormBD-blindspot CLI for library bias analysis
API endpoints: /publishers, /funding, /blindspots, /prompt-scores-multi

v0.5.0: Full Integration (Next)

Browser connector plugin
Desktop app integration
Real-time sync

License

PMPL-1.0-or-later

FormBD - The underlying narrative-first database engine
Zotero-Librarian - Filename normalization tool
Zotero - The original reference manager we build on top of
FormBD Debugger - Proof-carrying database debugger
FormBase - Open-source Airtable alternative with provenance

Name		Name	Last commit message	Last commit date
Latest commit History 22 Commits
migration		migration
src		src
.gitignore		.gitignore
LICENSE		LICENSE
Main.lean		Main.lean
README.adoc		README.adoc
install.ps1		install.ps1
install.sh		install.sh
lakefile.lean		lakefile.lean
lean-toolchain		lean-toolchain

License

hyperpolymath/zotero-formbd

Folders and files

Latest commit

History

Repository files navigation

Zotero-FormBD

Quick Start

Linux / macOS

Path A: I Already Use Zotero

Path B: I’m New to Reference Management

Step 1: Install Zotero (the UI)

Step 2: Install Julia (the engine)

Step 3: Open Zotero Once

Step 4: Install FormBD

Step 5: Daily Workflow

Windows

Prerequisites

Installation

Daily Workflow

Uninstall

What Do I Get?

Commands Reference

API Examples

Uninstall

Overview

Why Not Just SQLite?

Architecture

Core Types (Lean 4)

DOI Immutability (v0.3.0)

The Rules

Why This Matters

How It Works

Troubleshooting

"Cannot connect to Zotero" when running FormBD-sync

"Julia not found"

"Zotero database not found"

Roadmap

v0.1.0: Core Types + Journal ✓

v0.2.0: Evidence Quality ✓

v0.3.0: DOI Immutability ✓

v0.4.0: Source Intelligence ✓

v0.5.0: Full Integration (Next)

License

Related Projects

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors 2

Uh oh!

Languages

Packages