Advanced Windows Sandbox with Real-Time Host Intrusion Prevention System (HIPS)
Rusty Sand is a executable sandbox for Windows security research that i've made providing real-time API interception, behavioral threat detection, and interactive control over every operation a program performs. Unlike passive sandboxes that monitor after-the-fact, Rusty Sand implements a true HIPS that intercepts operations BEFORE execution using DLL injection and API hooking.
-
DLL Injection: Injects hook DLL into target process using
CreateRemoteThread -
MinHook Integration: Inline API hooking for Windows functions
-
Pre-Execution Blocking: Operations intercepted BEFORE they execute
-
Named Pipe IPC: Secure communication between main process and hook DLL
-
Modular Architecture: Hooks organized by category (file, folder, network, registry, process, memory) β¨ NEW
-
Hooked APIs (17 critical Windows functions - +112% coverage):
File Operations (2 hooks):
CreateFileW- File creation and modificationDeleteFileW- File deletion
Folder Operations (2 hooks):
CreateDirectoryW- Folder creationRemoveDirectoryW- Folder deletion
Network Operations (1 hook):
connect- Network connections (TCP/UDP)
Registry Operations (4 hooks):
RegSetValueExW- Registry value writesRegDeleteKeyW- Registry key deletionRegQueryValueExW- Registry value readsRegOpenKeyExW- Registry key opens
Process/Thread Operations (3 hooks) β¨ NEW:
CreateProcessW- Child process creationCreateThread- Thread creationCreateRemoteThread- Remote thread injection (CRITICAL for detecting process injection)
Memory/DLL Operations (5 hooks) β¨ NEW:
VirtualAlloc- Memory allocation (detects RWX allocations)VirtualProtect- Memory protection changes (detects DEP bypasses)WriteProcessMemory- Cross-process memory writes (detects code injection)LoadLibraryW- DLL loading (detects DLL injection)LoadLibraryExW- Extended DLL loading
- Intelligent Threat Assessment: Every intercepted operation analyzed in real-time
- 0-100 Risk Score: Quantitative threat rating based on multiple factors
- Threat Categorization: Four-tier classification system
- π’ LOW (0-30): Normal operations, minimal risk
- π‘ MEDIUM (31-60): Potentially suspicious, warrants attention
- π HIGH (61-85): Likely malicious, strong indicators
- π΄ CRITICAL (86-100): Almost certainly malicious, immediate action recommended
- Context-Aware Analysis: Scoring considers operation type, target location, parameters
- Smart Filtering: Auto-allows read-only operations to reduce prompt fatigue by ~60%
- Detection Patterns:
- Persistence: Registry Run keys (+65 risk), Startup folders (+60 risk)
- Code Injection: Remote thread creation (+95 risk), cross-process memory writes (+85 risk)
- DEP Bypass: RWX memory allocation (+65 risk), memory protection changes (+60 risk)
- Ransomware:
.encrypted/.lockedextensions (+70 risk), rapid file operations - UAC Bypass: Environment variable manipulation (+70 risk)
- Security Tampering: Windows Defender/Firewall modifications (+70 risk)
- C2 Communication: Suspicious ports 4444/31337 (+55 risk), large data exfiltration (+40 risk)
- Living-off-the-Land: PowerShell encoded commands (+50 risk), LOLBAS abuse (+40 risk)
- File System: File/folder creation, modification, deletion (real-time via API hooks)
- Network: TCP/UDP connections (intercepted before connect)
- Registry: Registry operations (intercepted before modification)
- Processes: Complete process tree tracking with Toolhelp32
Automatically detects:
- Ransomware: Rapid file encryption, suspicious extensions (
.encrypted,.locked) - Persistence: Registry Run keys, startup folders, scheduled tasks
- Process Injection: Remote thread creation, cross-process memory writes β¨ NEW
- Code Execution: RWX memory allocation, DEP bypass attempts β¨ NEW
- DLL Injection: Suspicious DLL loading from temp directories β¨ NEW
- UAC Bypass: Environment variable manipulation,
ms-settingsabuse - Security Tampering: Windows Defender/firewall modifications
- C2 Communications: Suspicious ports (4444, 8080, 31337), large data transfers β¨ NEW
- PowerShell Abuse: Encoded commands, download cradles, hidden window execution
- Folder Operations: Suspicious folder creation/deletion (ProgramData, System32)
- Windows Job Objects: Hard resource limits (memory, CPU)
- CREATE_SUSPENDED: Process starts suspended until user approval
- Process Tree Control: Suspend/resume using Toolhelp32 + threading APIs
- Network Isolation: Internet access OFF by default
- Dual Console: Target runs in separate window (clean monitoring UI)
- Color-coded Console Output: Events with icons and risk levels
- JSON Exports: Comprehensive machine-readable reports
- Event Statistics: Breakdown by operation type
- Threat Summaries: Detected behavioral patterns
Rusty Sand uses a sophisticated two-process architecture for real-time prevention:
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Main Process β
β (rusty_sand.exe) β
β β
β βββββββββββββββ ββββββββββββββββ ββββββββββββββββ β
β β Process β β Monitoring β β Behavioral β β
β β Controller β β Engine β β Analyzer β β
β βββββββββββββββ ββββββββββββββββ ββββββββββββββββ β
β β β β β
β ββββββββββββββββββββΌββββββββββββββββββββ β
β β β
β ββββββββββΌβββββββββ β
β β IPC Server β β
β β (Named Pipe) β β
β ββββββββββ¬βββββββββ β
βββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββ
β
\\.\pipe\rusty_sand_hooks
β
βββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββ
β Target Process β
β (suspended.exe) β
β β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β rusty_sand_hooks.dll (injected) β β
β β β β
β β βββββββββββββββ ββββββββββββββββ βββββββββββββ β β
β β β CreateFileWβ β connect() β βRegSetValueβ β β
β β β Hook β β Hook β β Hook β β β
β β βββββββββββββββ ββββββββββββββββ βββββββββββββ β β
β β β β β β β
β β βββββββββββββββββββΌβββββββββββββββββ β β
β β β β β
β β ββββββββββΌβββββββββ β β
β β β IPC Client β β β
β β β(request_approval) β β
β β βββββββββββββββββββ β β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Original Windows APIs (via MinHook trampolines) β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ-
Initialization
- Main process creates target in
CREATE_SUSPENDEDstate - IPC server starts on named pipe
\\.\pipe\rusty_sand_hooks - Hook DLL (
rusty_sand_hooks.dll) injected viaCreateRemoteThread - DLL connects to IPC server and installs MinHook hooks
- Main process creates target in
-
Initial Approval
- User prompted to allow initial execution
- If approved: process resumes, hooks are active
- If denied: process terminated immediately
-
Real-Time Interception
- Target calls
CreateFileW()β hook intercepts - Hook extracts parameters (file path, flags, attributes)
- Hook sends
HookRequestto main process via named pipe - Main process prompts user: Allow/Deny/Terminate
- Response sent back to hook DLL
- If allowed: call original API via trampoline
- If denied: return error handle WITHOUT calling original API
- Target calls
-
Shutdown
- Process exits (normally or terminated)
- Shutdown signal sent to all monitoring tasks
- 500ms grace period for cleanup
- Report generated
This is a Cargo workspace with two crates:
- rusty_sand - Main executable (binary crate)
- rusty_sand_hooks - Hook DLL (cdylib crate) β compiles to
rusty_sand_hooks.dll
Both must be built for full functionality. The DLL must be in the same directory as the executable.
- Windows 10/11 (x64)
- Rust toolchain (1.70+)
- Administrator privileges (recommended for full functionality)
# Clone the repository
git clone https://github.com/yourusername/rusty_sand.git
cd rusty_sand
# Build both main executable AND hook DLL (REQUIRED)
cargo build --release --workspace
# Binaries will be at:
# target\release\rusty_sand.exe
# target\release\rusty_sand_hooks.dll (must be in same directory!)Important: You must build the entire workspace. The hook DLL is required for API interception to work.
# Run with interactive HIPS mode (default)
.\target\release\rusty_sand.exe suspicious.exe
# Run in passive monitoring mode (no prompts, post-execution analysis only)
.\target\release\rusty_sand.exe --no-interactive malware.exe
# Enable internet access (β οΈ use with extreme caution!)
.\target\release\rusty_sand.exe --internet suspicious.exe
# Custom timeout and memory limits
.\target\release\rusty_sand.exe -t 60 -m 512 program.exe
# Pass arguments to sandboxed program
.\target\release\rusty_sand.exe program.exe -- arg1 arg2 arg3
# Verbose debug output
.\target\release\rusty_sand.exe -v suspicious.exe
# Disable API hooks (passive monitoring only)
.\target\release\rusty_sand.exe --no-interactive suspicious.exeEXECUTABLE- Path to executable to sandbox (absolute or relative)
-i, --internet- Enable internet access (β οΈ DEFAULT: DISABLED)-d, --dns- Enable DNS resolution-t, --timeout <SECONDS>- Execution timeout (default: 300)-m, --memory <MB>- Memory limit in MB (default: 1024)-w, --workdir <PATH>- Working directory for process-o, --output <DIR>- Output directory (default: ./sandbox_output)-f, --format <FORMAT>- Output format:console,json,both(default: both)-v, --verbose- Enable verbose debug output--log-network- Enable detailed network packet logging--no-registry- Disable registry monitoring--no-interactive- Disable HIPS mode (passive monitoring only)--no-behavior-detection- Disable behavioral threat detection
rusty_sand.exe --helpWhen running with API hooks enabled (default), Rusty Sand intercepts operations before execution and prompts in real-time with intelligent risk scoring:
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
β οΈ INTERCEPTED OPERATION #5
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
ACTION: CREATE REMOTE THREAD (Process Injection)
TARGET: explorer.exe (PID 1234)
DETAILS: Start address: 0x7FFE0000
RISK SCORE: π΄ 95/100 [CRITICAL]
π BLOCKED - Waiting for your decision...
[Y]es / [A]llow All / [N]o / [D]eny All / [T]erminate >
Risk scores help you make informed decisions:
- π’ LOW (0-30): Safe to allow, normal operations
- π‘ MEDIUM (31-60): Review carefully before allowing
- π HIGH (61-85): Suspicious, likely deny unless expected
- π΄ CRITICAL (86-100): Almost certainly malicious, deny recommended
When an operation is intercepted:
- [Y]es - Allow this single operation (calls original API)
- [A]llow All - Auto-allow ALL future operations of this type (no more prompts)
- [N]o - Block this single operation (returns error to target)
- [D]eny All - Auto-block ALL future operations of this type (no more prompts)
- [T]erminate - Immediately kill the entire process
If you choose [N]o, you'll see a follow-up prompt:
π« BLOCKED
[C]ontinue / [T]erminate >
- [C]ontinue - Continue monitoring the process
- [T]erminate - Kill the process immediately
βββββββββββββββββββββββββββββββββββββββββββββββββββ
SANDBOX EXECUTION REPORT
βββββββββββββββββββββββββββββββββββββββββββββββββββ
π EXECUTION DETAILS
Executable: suspicious.exe
Start Time: 2025-10-10 14:30:00 UTC
End Time: 2025-10-10 14:32:15 UTC
Duration: 135 seconds
Exit Code: 0
π SECURITY CONFIGURATION
Internet: DISABLED β
API Hooks: ENABLED (17 hooks active - +112% coverage) β¨
Risk Scoring: ENABLED (Real-time threat assessment) β¨
Interactive: ENABLED (HIPS mode)
Memory Limit: 1024 MB
π EVENT SUMMARY
Total Events: 68
File Operations: 12
Folder Operations: 3
Process Operations: 8 β¨
Memory Operations: 5 β¨
Network Blocked: 2 β οΈ
Registry Operations: 38
π¨ THREATS DETECTED (Risk Score)
[CRITICAL 95] Remote thread injection attempt in explorer.exe β¨
[CRITICAL 90] Registry persistence: HKLM\Run key modified
[HIGH 85] Cross-process memory write to svchost.exe β¨
[HIGH 75] RWX memory allocation (shellcode indicator) β¨
[MEDIUM 55] Suspicious folder creation in ProgramData
[MEDIUM 45] Attempted connection to C2 port 4444
π RECENT EVENTS (last 100)
π [14:30:05] FolderCreated: C:\ProgramData\Malware
π [14:30:08] FileCreated: C:\temp\output.txt
π [14:30:12] NetworkBlocked: TCP 192.168.1.100:4444
...
Saved to sandbox_output/report.json with complete event details, timestamps, user decisions, and configuration.
Rusty Sand can be used as a Rust library:
use rusty_sand::{execute_sandboxed, SandboxConfig};
use std::time::Duration;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let mut config = SandboxConfig::new()
.with_internet(false)
.with_timeout(Duration::from_secs(60))
.with_memory_limit(512)
.with_verbose(true);
// Enable API hooks for real-time prevention
config.enable_api_hooks = true;
config.interactive_mode = true;
let report = execute_sandboxed(
"suspicious.exe",
&["arg1".to_string(), "arg2".to_string()],
config
).await?;
println!("Total events: {}", report.events.len());
println!("Exit code: {}", report.exit_code);
// Access specific event types
let file_events = report.get_file_events();
let network_events = report.get_network_events();
// Analyze threats
for event in report.events {
if event.event_type == rusty_sand::report::EventType::FolderDeleted {
println!("Folder deleted: {}", event.details);
}
}
Ok(())
}See examples/basic_usage.rs and examples/advanced_monitoring.rs.
- Rapid file creation patterns (>50 files in short time)
- Suspicious file extensions (
.encrypted,.locked,.crypto) - Mass file deletion in user directories
- Registry Run key modifications (
HKCU\Software\Microsoft\Windows\CurrentVersion\Run) - Startup folder access
- Scheduled task creation
- Service installation
- Suspicious folder creation in
ProgramData(persistence) - Hidden folder creation (names starting with
.) - Critical system folder deletion (
System32,Program Files) - User data folder deletion (
Documents,Desktop,Downloads)
HKCU\Environment\windirmanipulationms-settingsprotocol abuse- DLL hijacking patterns
- C2 server communication (ports 4444, 8080, 31337)
- Blocked connections when internet disabled
- Unusual connection patterns
| Component | File | Purpose |
|---|---|---|
| Process Controller | src/control/mod.rs | Suspend/resume threads via Toolhelp32 |
| Interactive Controller | src/control/interactive.rs | HIPS prompts and user decisions |
| Monitoring Engine | src/monitor/mod.rs | Event collection, IPC server loop, risk analysis integration |
| File Monitor | src/monitor/filesystem.rs | Directory watching (notify crate) |
| Network Monitor | src/monitor/network.rs | TCP/UDP table polling |
| Process Monitor | src/monitor/process.rs | Process tree tracking |
| Risk Scorer β¨ NEW | src/analysis/risk_scorer.rs | Real-time threat scoring (0-100 scale) |
| Behavioral Analyzer | src/behavior/mod.rs | Threat pattern detection |
| Sandbox | src/sandbox/process.rs | Job objects, process creation |
| IPC Protocol | src/ipc/mod.rs | Named pipe communication |
| DLL Injection | src/injection/mod.rs | DLL injection via remote thread |
| Component | File | Purpose |
|---|---|---|
| Hook DLL Entry Point | rusty_sand_hooks/src/lib.rs | DLL initialization and MinHook orchestration |
| File Hooks | rusty_sand_hooks/src/hooks/file_hooks.rs | CreateFileW, DeleteFileW interception |
| Folder Hooks | rusty_sand_hooks/src/hooks/folder_hooks.rs | CreateDirectoryW, RemoveDirectoryW interception |
| Network Hooks | rusty_sand_hooks/src/hooks/network_hooks.rs | connect interception |
| Registry Hooks | rusty_sand_hooks/src/hooks/registry_hooks.rs | Registry operation interception (4 hooks) |
| Process Hooks β¨ NEW | rusty_sand_hooks/src/hooks/process_hooks.rs | Process/thread creation interception (3 hooks) |
| Memory Hooks β¨ NEW | rusty_sand_hooks/src/hooks/memory_hooks.rs | Memory/DLL operation interception (5 hooks) |
| IPC Types | rusty_sand_hooks/src/types.rs | Enhanced operation types with rich metadata |
| IPC Client | rusty_sand_hooks/src/ipc_client.rs | Named pipe communication layer |
| Registry Utils | rusty_sand_hooks/src/registry_utils.rs | HKEY-to-string conversion utilities |
| Logging System β¨ NEW | rusty_sand_hooks/src/logging.rs | Professional file-based logging with hook_log!() macro |
Hooked Functions (17 total - +112% increase):
- File Operations (2):
CreateFileW,DeleteFileW - Folder Operations (2):
CreateDirectoryW,RemoveDirectoryW - Network Operations (1):
connect - Registry Operations (4):
RegSetValueExW,RegDeleteKeyW,RegQueryValueExW,RegOpenKeyExW - Process/Thread Operations (3) β¨ NEW:
CreateProcessW,CreateThread,CreateRemoteThread - Memory/DLL Operations (5) β¨ NEW:
VirtualAlloc,VirtualProtect,WriteProcessMemory,LoadLibraryW,LoadLibraryExW
# Build main executable only
cargo build --release
# Build hook DLL only
cargo build --release --package rusty_sand_hooks
# Build entire workspace (recommended)
cargo build --release --workspace
# Debug build (faster compilation)
cargo build --workspace
# Check code without building
cargo check --workspace
# Run clippy linter
cargo clippy --workspace -- -D warnings
# Format code
cargo fmt --allrusty_sand/
βββ src/ # Main executable crate
β βββ analysis/ # Risk scoring system β¨ NEW
β β βββ mod.rs
β β βββ risk_scorer.rs # 0-100 threat scoring
β βββ behavior/ # Threat detection engine
β βββ control/ # Process control & HIPS
β βββ monitor/ # Monitoring subsystems
β βββ report/ # Reporting and output
β βββ sandbox/ # Process isolation
β βββ ipc/ # Named pipe IPC
β βββ injection/ # DLL injection
β βββ config.rs # Configuration
β βββ lib.rs # Library entry point
β βββ main.rs # CLI entry point
βββ rusty_sand_hooks/ # Hook DLL crate (modular architecture β¨ NEW)
β βββ src/
β β βββ hooks/ # Organized by category β¨ NEW
β β β βββ file_hooks.rs
β β β βββ folder_hooks.rs
β β β βββ network_hooks.rs
β β β βββ registry_hooks.rs
β β β βββ process_hooks.rs β¨ NEW
β β β βββ memory_hooks.rs β¨ NEW
β β βββ types.rs # Enhanced IPC types β¨ NEW
β β βββ ipc_client.rs # IPC communication β¨ NEW
β β βββ registry_utils.rs # HKEY utilities β¨ NEW
β β βββ logging.rs # Professional logging β¨ NEW
β β βββ utils.rs # Common utilities β¨ NEW
β β βββ lib.rs # DLL entry point (refactored)
β βββ Cargo.toml # DLL dependencies
βββ examples/ # Usage examples
βββ Cargo.toml # Workspace config
βββ EXECUTION_FLOW.md # Architecture documentation
βββ README.md
cargo run --release --example basic_usage
cargo run --release --example advanced_monitoringπ― Real-Time Risk Scoring System
- Intelligent 0-100 threat assessment for every intercepted operation
- Four-tier categorization (Low/Medium/High/Critical) with color-coded display
- Context-aware scoring considering operation type, target, and parameters
- Smart filtering auto-allows read-only operations (60% reduction in prompt fatigue)
π‘οΈ Expanded API Coverage (+112%)
- 17 hooked APIs (up from 8) - comprehensive protection coverage
- Process/Thread hooks: Detect process injection, remote thread creation
- Memory/DLL hooks: Catch RWX allocations, DEP bypasses, code injection
- Enhanced file/folder/registry/network interception with rich metadata
ποΈ Professional Modular Architecture
- Hook DLL refactored from 809-line monolith to organized module system
- Category-based organization (file, folder, network, registry, process, memory)
- Clean separation of concerns for maintainability
- File-based logging system with configurable levels and timestamps
π Enhanced Detection Capabilities
- Process Injection Detection:
CreateRemoteThread, cross-process memory writes - Code Execution Detection: RWX memory allocations, memory protection changes
- DLL Injection Detection: Suspicious library loading patterns
- Advanced Persistence: Comprehensive startup and autorun detection
- Living-off-the-Land: PowerShell abuse, LOLBAS detection
π Improved User Experience
- Risk scores displayed in real-time prompts with emoji indicators (π’π‘π π΄)
- Human-readable registry paths (
HKLM\Software\...instead of raw pointers) - Comprehensive operation metadata (access rights, share modes, protection flags)
- Professional logging for debugging hook DLL behavior
- Windows Only: Uses Win32 APIs exclusively (Job Objects, Toolhelp32, Named Pipes)
- User-Mode: Cannot intercept kernel-level operations or drivers
- Evasion: Sophisticated malware can detect hooks (MinHook inline hooking)
- Performance: HIPS mode significantly slows execution due to user prompts
- Admin Privileges: Some features require elevation
- Hook DLL Dependency: API interception requires successful DLL injection
Contributions welcome for:
- Additional API hooks (
WriteFile,NtCreateFile,GetProcAddress, etc.) - Enhanced risk scoring algorithms and detection patterns
- Behavioral detection rules (YARA-style, MITRE ATT&CK mapping)
- Performance optimizations (async hook handling, caching)
- Better Windows API integration (kernel callbacks, ETW tracing)
- Machine learning integration for risk scoring
- Bug fixes and stability improvements
- Documentation and examples
Please ensure all contributions are for defensive security purposes only.
MIT License - See LICENSE file for details.
For defensive security research only. The authors are not responsible for misuse.
- GuestAUser - Creator and primary developer
- MinHook - x86/x64 API hooking library
- Windows API documentation and community
- Rust security community
- Malware analysis research community
- Practical Malware Analysis: The Hands-On Guide to Dissecting Malicious Software 1st Edition by Michael Sikorski (Author), Andrew Honig (Author)
- Cuckoo Sandbox - Automated malware analysis
- CAPE Sandbox - Advanced malware analysis
- Sandboxie - Application sandboxing
- Any.Run - Interactive malware analysis service
Stay safe and sandbox everything! ποΈ
