Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,99 @@ let tasks: Vec<JoinHandle<Result<ExecutionResult>>> = nodes
- Buffered I/O for output collection
- Early termination on critical failures

**Signal Handling (Added 2025-12-16, Issue #95; Updated 2025-12-16, PR #102):**

The executor supports two modes for handling Ctrl+C (SIGINT) signals during parallel execution:

1. **Default Mode (Two-Stage)**:
- First Ctrl+C: Displays status (running/completed job counts)
- Second Ctrl+C (within 1 second): Terminates all jobs immediately with exit code 130
- Time window reset: If >1 second passes, next Ctrl+C restarts the sequence and shows status again
- Provides users visibility into execution progress before termination

2. **Batch Mode (`--batch` / `-b`)**:
- Single Ctrl+C: Immediately terminates all jobs with exit code 130
- Optimized for non-interactive environments (CI/CD, scripts)
- Compatible with pdsh `-b` option for tool compatibility

**Exit Code Handling:**
- Normal completion: Exit code determined by ExitCodeStrategy (MainRank/RequireAllSuccess/etc.)
- Signal termination (Ctrl+C): Always exits with code 130 (standard SIGINT exit code)
- This ensures scripts can detect user interruption vs. command failure

**Implementation Coverage:**
Signal handling is implemented in both execution modes:
- `execute()` method (normal/progress bar mode) - lines 172-280
- `handle_stream_mode()` method (stream mode) - lines 714-838
- TUI mode has its own quit handling (q or Ctrl+C) and ignores the batch flag

Implementation is in `executor/parallel.rs` using `tokio::select!` to handle signals alongside normal execution:

```rust
loop {
tokio::select! {
_ = signal::ctrl_c() => {
if self.batch {
// Batch mode: terminate immediately
eprintln!("\nReceived Ctrl+C (batch mode). Terminating all jobs...");
for handle in pending_handles.drain(..) {
handle.abort();
}
// Exit with SIGINT exit code (130)
std::process::exit(130);
} else {
// Two-stage mode: first shows status, second terminates
if !first_ctrl_c {
first_ctrl_c = true;
ctrl_c_time = Some(std::time::Instant::now());
eprintln!("\nReceived Ctrl+C. Press Ctrl+C again within 1 second to terminate.");

// Show status
let running_count = pending_handles.len();
let completed_count = self.nodes.len() - running_count;
eprintln!("Status: {} running, {} completed", running_count, completed_count);
} else {
// Second Ctrl+C: check time window
if let Some(first_time) = ctrl_c_time {
if first_time.elapsed() <= Duration::from_secs(1) {
// Within time window: terminate
eprintln!("Received second Ctrl+C. Terminating all jobs...");
for handle in pending_handles.drain(..) {
handle.abort();
}
// Exit with SIGINT exit code (130)
std::process::exit(130);
} else {
// Time window expired: reset and show status again
first_ctrl_c = true;
ctrl_c_time = Some(std::time::Instant::now());
eprintln!("\nReceived Ctrl+C. Press Ctrl+C again within 1 second to terminate.");

// Show current status
let running_count = pending_handles.len();
let completed_count = self.nodes.len() - running_count;
eprintln!("Status: {} running, {} completed", running_count, completed_count);
}
}
}
}
}
// Wait for all tasks to complete
results = join_all(pending_handles.iter_mut()) => {
return self.collect_results(results);
}
}

// Small sleep to avoid busy waiting
tokio::time::sleep(Duration::from_millis(50)).await;
}
```

The batch flag is passed through the executor chain:
- CLI `--batch` flag → `ExecuteCommandParams.batch` → `ParallelExecutor.batch`
- Applied in both normal mode (`execute()`) and stream mode (`handle_stream_mode()`)
- TUI mode maintains its own quit handling and ignores this flag

### 4. SSH Client (`ssh/client/*`, `ssh/tokio_client/*`)

**SSH Client Module Structure (Refactored 2025-10-17):**
Expand Down
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,32 @@ bssh -C production "df -h" > disk-usage.log
CI=true bssh -C production "command"
```

### Batch Mode (Ctrl+C Handling)

bssh provides two modes for handling Ctrl+C during parallel execution:

**Default (Two-Stage)**:
- First Ctrl+C: Shows status (running/completed counts)
- Second Ctrl+C (within 1 second): Terminates all jobs

**Batch Mode (`-b` / `--batch`)**:
- Single Ctrl+C: Immediately terminates all jobs
- Useful for non-interactive scripts and CI/CD pipelines

```bash
# Default behavior (two-stage Ctrl+C)
bssh -C production "long-running-command"
# Ctrl+C once: shows status
# Ctrl+C again (within 1s): terminates

# Batch mode (immediate termination)
bssh -C production -b "long-running-command"
# Ctrl+C once: immediately terminates all jobs

# Useful for automation
bssh -H nodes --batch --stream "deployment-script.sh"
```

### Built-in Commands
```bash
# Test connectivity to hosts
Expand Down
1 change: 1 addition & 0 deletions src/app/dispatcher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,7 @@ async fn handle_exec_command(cli: &Cli, ctx: &AppContext, command: &str) -> Resu
require_all_success: cli.require_all_success,
check_all_nodes: cli.check_all_nodes,
sudo_password,
batch: cli.batch,
};
execute_command(params).await
}
Expand Down
9 changes: 8 additions & 1 deletion src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ use std::path::PathBuf;
before_help = "\n\nBroadcast SSH - Parallel command execution across cluster nodes",
about = "Broadcast SSH - SSH-compatible parallel command execution tool",
long_about = "bssh is a high-performance SSH client with parallel execution capabilities.\nIt can be used as a drop-in replacement for SSH (single host) or as a powerful cluster management tool (multiple hosts).\n\nThe tool provides secure file transfer using SFTP and supports SSH keys, SSH agent, and password authentication.\nIt automatically detects Backend.AI multi-node session environments.\n\nOutput Modes:\n- TUI Mode (default): Interactive terminal UI with real-time monitoring (auto-enabled in terminals)\n- Stream Mode (--stream): Real-time output with [node] prefixes\n- File Mode (--output-dir): Save per-node output to timestamped files\n- Normal Mode: Traditional output after all nodes complete\n\nSSH Configuration Support:\n- Reads standard SSH config files (defaulting to ~/.ssh/config)\n- Supports Host patterns, HostName, User, Port, IdentityFile, StrictHostKeyChecking\n- ProxyJump, and many other SSH configuration directives\n- CLI arguments override SSH config values following SSH precedence rules",
after_help = "EXAMPLES:\n SSH Mode:\n bssh user@host # Interactive shell\n bssh admin@server.com \"uptime\" # Execute command\n bssh -p 2222 -i ~/.ssh/key user@host # Custom port and key\n bssh -F ~/.ssh/myconfig webserver # Use custom SSH config\n\n Port Forwarding:\n bssh -L 8080:example.com:80 user@host # Local forward: localhost:8080 → example.com:80\n bssh -R 8080:localhost:80 user@host # Remote forward: remote:8080 → localhost:80\n bssh -D 1080 user@host # SOCKS5 proxy on localhost:1080\n bssh -L 3306:db:3306 -R 80:web:80 user@host # Multiple forwards\n bssh -D *:1080/4 user@host # SOCKS4 proxy on all interfaces\n\n Multi-Server Mode:\n bssh -C production \"systemctl status\" # Execute on cluster (TUI mode auto-enabled)\n bssh -H \"web1,web2,web3\" \"df -h\" # Execute on multiple hosts\n bssh -H \"web1,web2,web3\" -f \"web1\" \"df -h\" # Filter to web1 only\n bssh -C production -f \"web*\" \"uptime\" # Filter cluster nodes\n bssh --parallel 20 -H web* \"apt update\" # Increase parallelism\n\n Host Exclusion (--exclude):\n bssh -H \"node1,node2,node3\" --exclude \"node2\" \"uptime\" # Exclude single host\n bssh -C production --exclude \"web1,web2\" \"apt update\" # Exclude multiple hosts\n bssh -C production --exclude \"db*\" \"systemctl restart\" # Exclude with wildcard pattern\n bssh -C production --exclude \"*-backup\" \"df -h\" # Exclude backup nodes\n\n Output Modes:\n bssh -C prod \"apt-get update\" # TUI mode (default, interactive monitoring)\n bssh -C prod --stream \"tail -f log\" # Stream mode (real-time with [node] prefixes)\n bssh -C prod --output-dir ./logs \"ps\" # File mode (save to timestamped files)\n bssh -C prod \"uptime\" | tee log.txt # Normal mode (auto-detected when piped)\n\n TUI Mode Controls (when in TUI):\n 1-9 Jump to node detail view\n s Enter split view (2-4 nodes)\n d Enter diff view (compare nodes)\n f Toggle auto-scroll\n ↑/↓ Scroll output\n ←/→ Switch nodes\n Esc Return to summary\n ? Show help\n q Quit\n\n File Operations:\n bssh -C staging upload file.txt /tmp/ # Upload to cluster\n bssh -H host1,host2 download /etc/hosts ./backups/\n\n Other Commands:\n bssh list # List configured clusters\n bssh -C production ping # Test connectivity\n bssh -H hosts interactive # Interactive mode\n\n SSH Config Example (~/.ssh/config):\n Host web*\n HostName web.example.com\n User webuser\n Port 2222\n IdentityFile ~/.ssh/web_key\n StrictHostKeyChecking yes\n\nDeveloped and maintained as part of the Backend.AI project.\nFor more information: https://github.com/lablup/bssh"
after_help = "EXAMPLES:\n SSH Mode:\n bssh user@host # Interactive shell\n bssh admin@server.com \"uptime\" # Execute command\n bssh -p 2222 -i ~/.ssh/key user@host # Custom port and key\n bssh -F ~/.ssh/myconfig webserver # Use custom SSH config\n\n Port Forwarding:\n bssh -L 8080:example.com:80 user@host # Local forward: localhost:8080 → example.com:80\n bssh -R 8080:localhost:80 user@host # Remote forward: remote:8080 → localhost:80\n bssh -D 1080 user@host # SOCKS5 proxy on localhost:1080\n bssh -L 3306:db:3306 -R 80:web:80 user@host # Multiple forwards\n bssh -D *:1080/4 user@host # SOCKS4 proxy on all interfaces\n\n Multi-Server Mode:\n bssh -C production \"systemctl status\" # Execute on cluster (TUI mode auto-enabled)\n bssh -H \"web1,web2,web3\" \"df -h\" # Execute on multiple hosts\n bssh -H \"web1,web2,web3\" -f \"web1\" \"df -h\" # Filter to web1 only\n bssh -C production -f \"web*\" \"uptime\" # Filter cluster nodes\n bssh --parallel 20 -H web* \"apt update\" # Increase parallelism\n\n Host Exclusion (--exclude):\n bssh -H \"node1,node2,node3\" --exclude \"node2\" \"uptime\" # Exclude single host\n bssh -C production --exclude \"web1,web2\" \"apt update\" # Exclude multiple hosts\n bssh -C production --exclude \"db*\" \"systemctl restart\" # Exclude with wildcard pattern\n bssh -C production --exclude \"*-backup\" \"df -h\" # Exclude backup nodes\n\n Output Modes:\n bssh -C prod \"apt-get update\" # TUI mode (default, interactive monitoring)\n bssh -C prod --stream \"tail -f log\" # Stream mode (real-time with [node] prefixes)\n bssh -C prod --output-dir ./logs \"ps\" # File mode (save to timestamped files)\n bssh -C prod \"uptime\" | tee log.txt # Normal mode (auto-detected when piped)\n\n Batch Mode (Ctrl+C Handling):\n bssh -C prod \"long-running-command\" # Default: first Ctrl+C shows status, second terminates\n bssh -C prod -b \"long-command\" # Batch mode: single Ctrl+C terminates immediately\n bssh -H nodes --batch --stream \"cmd\" # Useful for CI/CD and non-interactive scripts\n\n TUI Mode Controls (when in TUI):\n 1-9 Jump to node detail view\n s Enter split view (2-4 nodes)\n d Enter diff view (compare nodes)\n f Toggle auto-scroll\n ↑/↓ Scroll output\n ←/→ Switch nodes\n Esc Return to summary\n ? Show help\n q Quit\n\n File Operations:\n bssh -C staging upload file.txt /tmp/ # Upload to cluster\n bssh -H host1,host2 download /etc/hosts ./backups/\n\n Other Commands:\n bssh list # List configured clusters\n bssh -C production ping # Test connectivity\n bssh -H hosts interactive # Interactive mode\n\n SSH Config Example (~/.ssh/config):\n Host web*\n HostName web.example.com\n User webuser\n Port 2222\n IdentityFile ~/.ssh/web_key\n StrictHostKeyChecking yes\n\nDeveloped and maintained as part of the Backend.AI project.\nFor more information: https://github.com/lablup/bssh"
)]
pub struct Cli {
/// SSH destination in format: [user@]hostname[:port] or ssh://[user@]hostname[:port]
Expand Down Expand Up @@ -104,6 +104,13 @@ pub struct Cli {
)]
pub sudo_password: bool,

#[arg(
short = 'b',
long = "batch",
help = "Batch mode: single Ctrl+C immediately terminates all jobs\nDisables two-stage Ctrl+C handling (status display on first press)\nUseful for non-interactive scripts and CI/CD pipelines\nNote: TUI mode has its own quit handling (q or Ctrl+C) and ignores this flag"
)]
pub batch: bool,

#[arg(
short = 'J',
long = "jump-host",
Expand Down
5 changes: 4 additions & 1 deletion src/commands/exec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ pub struct ExecuteCommandParams<'a> {
pub require_all_success: bool,
pub check_all_nodes: bool,
pub sudo_password: Option<Arc<SudoPassword>>,
pub batch: bool,
}

pub async fn execute_command(params: ExecuteCommandParams<'_>) -> Result<()> {
Expand Down Expand Up @@ -174,6 +175,7 @@ async fn execute_command_with_forwarding(params: ExecuteCommandParams<'_>) -> Re
// Execute the actual command
let result = execute_command_without_forwarding(ExecuteCommandParams {
port_forwards: None, // Remove forwarding from params to avoid recursion
batch: params.batch,
..params
})
.await;
Expand Down Expand Up @@ -209,7 +211,8 @@ async fn execute_command_without_forwarding(params: ExecuteCommandParams<'_>) ->
.with_timeout(params.timeout)
.with_connect_timeout(params.connect_timeout)
.with_jump_hosts(params.jump_hosts.map(|s| s.to_string()))
.with_sudo_password(params.sudo_password);
.with_sudo_password(params.sudo_password)
.with_batch_mode(params.batch);

// Set keychain usage if on macOS
#[cfg(target_os = "macos")]
Expand Down
Loading
Loading