Arc Helper Scripts

A collection of Python scripts for working with Arc Timeline and Arc Editor backup data. Includes tools for filtering, analyzing, and exporting location timeline data.

Important: Different tools support different backup formats:

• Filter tool → LocoKit1 & LocoKit2 formats (auto-detects backup structure)
• Analysis tools → LocoKit2 format only (Arc Editor backup with items/ folders)

Overview

This toolkit provides four complementary utilities for Arc/LocoKit backup analysis:

1. Filter Backup by Date Range (LocoKit1 & LocoKit2) - Creates filtered backup copies for specific time periods
2. Analyze Place Sample Distribution (LocoKit2) - Examines GPS sample accuracy for place visits
3. Check Visit-Place Distance (LocoKit2) - Identifies visits far from their assigned places
4. Export Visit Samples (LocoKit2) - Exports all GPS samples for a specific visit

Quick Start

All scripts support three interfaces: Windows Batch (double-click), PowerShell, or Python CLI.

1. Filter Backup by Date Range

Extract timeline data for a specific time period:

Windows Batch:

filter_backup.bat

PowerShell:

.\filter_backup_by_daterange.ps1 -BackupDir "c:\backup" -Days 7

Python:

python filter_backup_by_daterange.py --backup-dir "c:\backup" --days 7

2. Analyze Place Sample Distribution

Examine GPS sample accuracy and outliers for a place:

Windows Batch:

analyze_place_samples.bat

PowerShell:

.\analyze_place_samples.ps1 -PlaceName "Home" -MaxOutliers 30

Python:

python analyze_place_samples.py "Home"

3. Check Visit-Place Distance

Find visits that are far from their assigned place:

Windows Batch:

check_visit_place_distance.bat

PowerShell:

.\check_visit_place_distance.ps1 -Threshold 100

Python:

python check_visit_place_distance.py --threshold 100

4. Export Visit Samples

Export all GPS samples for a visit to JSON:

Windows Batch:

export_visit_samples.bat

PowerShell:

.\export_visit_samples.ps1 -VisitId "4DC38EA8-2CC8-4358-A912-AA4D2BECF458"

Python:

python export_visit_samples.py 4DC38EA8-2CC8-4358-A912-AA4D2BECF458

Features

✅ Four specialized tools for Arc backup analysis
✅ Multiple interfaces: Batch, PowerShell, and Python CLI
✅ LocoKit1 & LocoKit2 support: Filter tool supports both formats, analysis tools for LocoKit2
✅ Minimal dependencies: Pure Python 3.7+, no external packages required
✅ Parallel processing: Efficient for large backups
✅ Windows-native: Batch and PowerShell scripts included
✅ Cross-platform: Python core works on Windows, macOS, Linux

Installation

Requirements

• Python 3.7+
• A valid Arc backup directory with appropriate structure:
  • LocoKit1 (for filter tool): TimelineItem/, LocomotionSample/, Place/ folders
  • LocoKit2 (for analysis tools): items/, samples/, places/ folders

Setup

1. Ensure Python is in your PATH:

   python --version

2. Clone or download this repository

3. Run any script using your preferred interface:

   • Batch: Double-click .bat files
   • PowerShell: .\script_name.ps1
   • Python: python script_name.py --help

No external dependencies or pip installs required!

Detailed Usage

1. Filter Backup by Date Range

Creates a filtered copy of a LocoKit1 or LocoKit2 backup containing only data within a specified date range. Automatically detects backup format and maintains complete directory structure.

Use cases:

• Creating test datasets
• Analyzing specific time periods
• Reducing backup size for sharing
• Creating period-specific archives

Examples:

# Filter by date range
python filter_backup_by_daterange.py `
  --backup-dir "C:\backup" `
  --start "2024-12-15 00:00:00" `
  --end "2024-12-31 23:59:59" `
  --output-dir "C:\filtered_backup"

# Filter last 7 days
python filter_backup_by_daterange.py --backup-dir "C:\backup" --days 7

# Filter single day
python filter_backup_by_daterange.py --backup-dir "C:\backup" --date "2024-12-25"

Options:

--backup-dir BACKUP_DIR   Path to LocoKit1 or LocoKit2 backup root directory (required)
--output-dir OUTPUT_DIR   Output directory (default: ./filtered_backup)
--start START             Start date/time (YYYY-MM-DD HH:MM:SS)
--end END                 End date/time (YYYY-MM-DD HH:MM:SS)
--date DATE               Single date to filter (YYYY-MM-DD)
--days DAYS               Number of days back from today

Output:

LocoKit1 structure:

filtered_backup/
├── TimelineItem/      # Hex-bucketed (00-FF)
├── LocomotionSample/  # Week-bucketed .json.gz
└── Place/             # Hex-bucketed

LocoKit2 structure:

filtered_backup/
├── items/             # Monthly JSON arrays
├── samples/           # Week-bucketed .json.gz
└── places/            # Bucketed JSON arrays

---

2. Analyze Place Sample Distribution

Analyzes GPS sample distribution for visits to a specific place. Reports distance statistics and identifies outlier samples.

Use cases:

• Identifying GPS drift issues
• Verifying place radius accuracy
• Finding visits with poor location data
• Quality checking place definitions

Examples:

# Analyze by place name
python analyze_place_samples.py "Home"

# Analyze by place ID
python analyze_place_samples.py --place-id 1A5FC6C4-0E29-4653-B52C-0604AF01A5DC

# Show more outliers
python analyze_place_samples.py "Home" --max-outliers 50

# Include deleted/disabled items
python analyze_place_samples.py "Office" --include-inactive

Options:

--place PLACE              Place name to search for
--place-id PLACE_ID        Exact place ID (UUID)
--backup-dir BACKUP_DIR    Path to Arc Editor backup directory
--include-inactive         Include deleted/disabled/locked visits and samples
--max-outliers N           Number of outlier samples to show (default: 20)
--max-samples N            Stop after N matched samples (for testing)

Output:

• Distance statistics (min, mean, median, P90, P95, P99, max)
• Weighted center calculation with accuracy weighting
• Distribution by distance bins
• Top sample outliers with coordinates and timestamps
• Visits with maximum sample distances

---

3. Check Visit-Place Distance

Identifies visits where the visit location is far from the assigned place center. Helps find misassigned or incorrectly geocoded visits.

Use cases:

• Finding incorrectly assigned visits
• Identifying GPS issues during visits
• Quality checking place assignments
• Detecting place center inaccuracies

Examples:

# Check with default 100m threshold
python check_visit_place_distance.py

# Use custom threshold
python check_visit_place_distance.py --threshold 50

# Include inactive visits
python check_visit_place_distance.py --threshold 500 --include-inactive

Options:

--threshold N              Distance threshold in meters (default: 100)
--backup-dir BACKUP_DIR    Path to Arc Editor backup directory
--include-inactive         Include deleted/disabled/locked visits

Output:

• Overall distance statistics
• List of problematic visits sorted by distance
• Visit details including duration, coordinates, health data
• Distribution by distance ranges

---

4. Export Visit Samples

Exports all GPS samples (LocomotionSample records) for a specific visit ID to a JSON file for detailed analysis.

Use cases:

• Analyzing GPS accuracy for specific visits
• Debugging timeline issues
• Extracting data for external analysis tools
• Creating sample datasets for testing

Examples:

# Export to default location
python export_visit_samples.py 4DC38EA8-2CC8-4358-A912-AA4D2BECF458

# Specify output path
python export_visit_samples.py ^
  --visit-id 4DC38EA8-2CC8-4358-A912-AA4D2BECF458 ^
  --output "visit_samples.json"

Options:

--visit-id VISIT_ID        Visit ID (UUID) to export samples for (required)
--backup-dir BACKUP_DIR    Path to Arc Editor backup directory
--output OUTPUT            Output JSON file path

Output:

• JSON array of all LocomotionSample records for the visit
• Includes coordinates, timestamps, accuracy, source, and more
• Default output: data_exports/visit_{visit_id}_samples.json

Common Use Cases

Debugging Timeline Issues

1. Use check_visit_place_distance.py to find problematic visits
2. Use export_visit_samples.py to get detailed GPS data for those visits
3. Use analyze_place_samples.py to check overall place accuracy

Creating Test Datasets

1. Use filter_backup_by_daterange.py to extract a date range
2. Share the filtered backup for testing or development
3. Use other tools to verify data quality

Quality Analysis Workflow

1. Identify a place with accuracy concerns
2. Run analyze_place_samples.py for sample distribution
3. Check specific visits with export_visit_samples.py
4. Fix place definitions or visit assignments as needed

Troubleshooting

Issue	Solution
Python not found	Install Python 3.7+ from https://www.python.org and select "Add Python to PATH"
"places directory not found"	Using LocoKit2 tool on LocoKit1 backup - use correct backup format
"TimelineItem directory not found"	Using LocoKit1 tool on LocoKit2 backup - use correct backup format
PowerShell execution error	Run Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope Process
No data in output	Try wider date range or lower threshold; verify backup has data
Out of disk space	Ensure output location has sufficient free space
Script runs slowly	Normal for large backups; use --max-samples for testing

Date Format (filter script)

Use ISO format with time: YYYY-MM-DD HH:MM:SS

Examples:

• 2024-12-15 00:00:00 — Start of Dec 15, 2024
• 2024-12-31 23:59:59 — End of Dec 31, 2024
• 2025-02-13 14:30:45 — 2:30:45 PM on Feb 13, 2025

Notes

• Original backups are never modified; all operations are read-only
• Empty/corrupted files are skipped with warnings logged
• Backup format compatibility:
  • Filter tool (filter_backup_by_daterange.py) supports both LocoKit1 and LocoKit2 with auto-detection
  • Analysis tools (analyze, check, export) require LocoKit2 format
• Places are only copied (filter script) if referenced by filtered items
• All scripts use standard library only—no external dependencies required

Technical Details

Supported Backup Formats

LocoKit1 (Arc v3):

• TimelineItem: Hex-bucketed JSON files (00-FF)
• LocomotionSample: ISO-week-bucketed gzip JSON files (YYYY-Wnn.json.gz)
• Place: Hex-bucketed place metadata

LocoKit2 (Arc Editor):

• items: Month-bucketed JSON files (YYYY-MM.json)
• samples: Week-bucketed gzip JSON files (YYYY-Wnn.json.gz)
• places: Hex-bucketed JSON files

Python Version Support

• Python 3.7+
• Uses only standard library (no external dependencies)
• Compatible with Windows, macOS, and Linux

Parallel Processing

• Filter script: 16 parallel workers for TimelineItem buckets
• Analysis scripts: Sequential file processing with progress reporting
• Efficient I/O handling for large backups

Distance Calculations

All distance calculations use the Haversine formula for great circle distance on Earth:

• Input: Decimal degree coordinates (latitude, longitude)
• Output: Meters
• Earth radius: 6,371,000 meters

Script Reference

Script	Purpose	Backup Format	Input	Output
filter_backup_by_daterange.py	Filter backup by date	LocoKit1 & LocoKit2	Backup + date range	Filtered backup copy
analyze_place_samples.py	Analyze GPS sample distribution	LocoKit2 only	Backup + place name/ID	Statistical report
check_visit_place_distance.py	Find visits far from places	LocoKit2 only	Backup + threshold	Distance analysis report
export_visit_samples.py	Export visit GPS samples	LocoKit2 only	Backup + visit ID	JSON file of samples

Contributing

Found a bug or want to suggest a feature? Please use the Issues tab on GitHub.

License

Apache License 2.0 — See LICENSE file for details

Attribution

Created with GitHub Copilot.

Acknowledgement

Big thanks to Matt Greenfield (@sobri909) for creating Arc and LocoKit in the first place.

Related

• Arc App — Location timeline tracking app
• LocoKit1 — Original Arc location processing library
• LocoKit2 — Next generation location processing library

---

Version: 2.0
Last Updated: February 2026
Tested on: Windows 11, Python 3.11