Skip to content

Official DataNodes Upload SDK - Multi-platform chunked uploads for maximum speed

License

Notifications You must be signed in to change notification settings

DataNodes-To/upload-sdk

Repository files navigation

DataNodes Upload SDK

Official multi-platform upload SDK for DataNodes - a file hosting service.

Looking for pre-compiled binaries? Download ready-to-run executables from upload-binaries - no dependencies required!

Features:

  • Multi-threaded chunked uploads for maximum speed (800+ MB/s)
  • Automatic retry with exponential backoff
  • Batch file and folder uploads
  • Progress bar with speed and ETA
  • Cross-platform support

Performance

SDK Speed Notes
Go ~885 MB/s Fastest, single binary
Node.js ~840 MB/s Uses worker threads
Python ~930 MB/s ThreadPoolExecutor
Bash ~900 MB/s Uses curl + parallel
PHP ~800 MB/s curl_multi
PowerShell ~750 MB/s Runspace pools

Speeds measured uploading 20GB file on 1Gbps connection

Quick Start

Bash

# Requirements: curl, jq, bc
./datanodes-upload.sh myfile.zip YOUR_API_KEY

Python

# Requirements: Python 3.6+, requests
pip install requests
python3 datanodes-upload.py myfile.zip YOUR_API_KEY

Node.js

# Requirements: Node.js 14+, axios
npm install axios
node datanodes-upload.js myfile.zip YOUR_API_KEY

Go

# Requirements: Go 1.18+ (no dependencies)
go run datanodes-upload.go myfile.zip YOUR_API_KEY

# Or build binary
go build -o datanodes-upload datanodes-upload.go
./datanodes-upload myfile.zip YOUR_API_KEY

PHP

# Requirements: PHP 7.4+ with curl extension
php datanodes-upload.php myfile.zip YOUR_API_KEY

PowerShell

# Requirements: PowerShell 5.1+ (Windows) or 7+ (cross-platform)
.\datanodes-upload.ps1 -Files myfile.zip -ApiKey YOUR_API_KEY

Common Options

All SDKs support these options:

Option Description Default
-t, --threads Parallel upload threads 10
-r, --retries Retries per chunk 3
-f, --folder Destination folder ID 0 (root)
-d, --directory Upload entire directory -
-R, --recursive Include subdirectories off

Examples

Upload with custom settings

# 20 threads, 5 retries, to folder ID 123
./datanodes-upload.sh -t 20 -r 5 -f 123 myfile.zip YOUR_API_KEY

Batch upload multiple files

./datanodes-upload.sh file1.zip file2.zip file3.zip YOUR_API_KEY

Upload entire folder

./datanodes-upload.sh -d /path/to/folder YOUR_API_KEY

Upload folder recursively

./datanodes-upload.sh -d /path/to/folder -R YOUR_API_KEY

Getting Your API Key

  1. Log in to DataNodes
  2. Go to Account > API Keys
  3. Copy your API key

Finding Folder ID

To upload to a specific folder:

  1. Navigate to the folder on DataNodes
  2. The folder ID is in the URL: https://datanodes.to/files?fld_id=123
  3. Use -f 123 when uploading

Output Example

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃   ✓ UPLOAD COMPLETE                                           ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

  Summary
  ─────────────────────────────────────────────────
  File:          myfile.zip
  Size:          19.53 GB
  Upload Time:   22.6s
  Avg Speed:     885.36 MB/s

  Links
  ─────────────────────────────────────────────────
  Download: https://datanodes.to/abc123xyz/myfile.zip
  Delete:   https://datanodes.to/abc123xyz/myfile.zip?killcode=xyz789

Documentation

Detailed documentation for each SDK:

How It Works

  1. Get Upload Server - Requests dedicated upload server from API
  2. Generate Session ID - Creates unique 16-digit session identifier
  3. Chunk File - Splits file into 1-100 MB chunks
  4. Parallel Upload - Uploads chunks simultaneously (10 threads default)
  5. Retry Failed - Automatically retries failed chunks with backoff
  6. Finalize - Imports chunks into final file on server

API Reference

Get Upload Server

GET https://datanodes.to/api/upload/server?key=API_KEY

Upload Chunk

PUT https://server.datanodes.to/upload/put_chunk_mt.cgi
Headers:
  X-Upload-SID: <session_id>
  X-Seek-To: <byte_offset>
Body: <chunk_data>

Finalize Upload

POST https://server.datanodes.to/upload/import_file
Body: fn=<filename>&fld_id=<folder>&st=OK&op=upload_result&sess_id=<session_id>

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License - see LICENSE file.

Support