Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 5 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ Options:
#### Store/fetch private data

```bash
# Store a file directly
# Store a the file `data.txt` under the name `mykey`
$> mutant put mykey data.txt

# Get a value and save to a file
Expand Down Expand Up @@ -256,7 +256,7 @@ Add `mutant-lib` and its dependencies to your `Cargo.toml`:

```toml
[dependencies]
mutant-lib = "0.6.1" # Or the version you need
mutant-lib = "0.6.2" # Or the version you need
tokio = { version = "1", features = ["full"] }
```

Expand All @@ -277,7 +277,6 @@ async fn main() -> anyhow::Result<()> {
mutant.put("greeting", b"hello world", StorageMode::Medium, false).await?;

let fetched_value = mutant.get("greeting").await?;

println!("Fetched value: {}", String::from_utf8_lossy(&fetched_value));

mutant.rm("greeting").await?;
Expand Down Expand Up @@ -353,45 +352,8 @@ async fn main() -> Result<()> {
}
```

## Development and Testing

### Local Testnet Management (`scripts/manage_local_testnet.sh`)

### Running Integration Tests (`scripts/run_tests_with_env.sh`)

## Migration

## Architecture Overview

MutAnt consists of five main components that work together to provide a complete storage solution:

1. **mutant-lib**: Core library handling chunking, encryption, and storage operations
2. **mutant-protocol**: Shared communication format definitions
3. **mutant-daemon**: Background service maintaining Autonomi connection
4. **mutant-client**: WebSocket client library for communicating with the daemon
5. **mutant-cli**: Command-line interface for end users

These components work together in a client-server architecture:

- The **daemon** uses **mutant-lib** to interact with the Autonomi network
- **Clients** (CLI or custom applications) connect to the daemon via WebSocket
- Communication between clients and the daemon uses the protocol definitions
- The daemon manages concurrent operations, background tasks, and network connections

This architecture provides several benefits:
- Persistent connection to the Autonomi network
- Background processing of long-running operations
- Task management and monitoring
- Concurrent operations with efficient resource usage
- Worker pools for optimized performance

### Internal Architecture
# Donate

Under the hood, MutAnt uses a worker architecture for handling operations:
- 10 workers, each with a dedicated client
- Each worker manages 10 concurrent operations (tasks)
- Total of 100 concurrent operations
- Round-robin distribution of work with work stealing
- Automatic recycling of failed pads
If you find this project useful, you can donate to support its development. <3

## Configuration
ETH/ANT `0x3376C33FdC0c885cef483690ffDd1e0Ff0Eb026c`
6 changes: 3 additions & 3 deletions mutant-cli/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "mutant"
version = "0.6.1"
version = "0.6.2"
edition = "2021"
license = "LGPL-3.0-only"
description = "Command-line interface for MutAnt distributed mutable key value storage over Autonomi network"
Expand All @@ -13,8 +13,8 @@ colored = "2.0"
env_logger = "0.11"
indicatif = "0.17"
log = "0.4"
mutant-client = { path = "../mutant-client", version = "0.6.1" }
mutant-protocol = { path = "../mutant-protocol", version = "0.6.1" }
mutant-client = { path = "../mutant-client", version = "0.6.2" }
mutant-protocol = { path = "../mutant-protocol", version = "0.6.2" }
tokio = { version = "1.0", features = ["full"] }
uuid = { version = "1.0", features = ["v4", "serde"] }
humansize = "2.1"
Expand Down
50 changes: 6 additions & 44 deletions mutant-cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ $> cargo install mutant
You can fetch the daily meme of the day with the following command:

```bash
$> mutant get -p 9620303d41cd65177e6763809f4780b1fa7d864a14d4ed0ed0322c5d4524fe406db65fdc485f1737814c81ab6d61dab0 daily_meme.jpg
$> mutant get -p a420224971527d61ce6ee21d850a07c243498c95808697e8fac23f461545656933016697d10b805c0fa26b50eb3532b2 daily_meme.jpg
```

## Command-Line Interface (CLI)
Expand Down Expand Up @@ -169,7 +169,7 @@ Options:
#### Store/fetch private data

```bash
# Store a file directly
# Store a the file `data.txt` under the name `mykey`
$> mutant put mykey data.txt

# Get a value and save to a file
Expand Down Expand Up @@ -256,7 +256,7 @@ Add `mutant-lib` and its dependencies to your `Cargo.toml`:

```toml
[dependencies]
mutant-lib = "0.6.0" # Or the version you need
mutant-lib = "0.6.2" # Or the version you need
tokio = { version = "1", features = ["full"] }
```

Expand All @@ -277,7 +277,6 @@ async fn main() -> anyhow::Result<()> {
mutant.put("greeting", b"hello world", StorageMode::Medium, false).await?;

let fetched_value = mutant.get("greeting").await?;

println!("Fetched value: {}", String::from_utf8_lossy(&fetched_value));

mutant.rm("greeting").await?;
Expand Down Expand Up @@ -353,45 +352,8 @@ async fn main() -> Result<()> {
}
```

## Development and Testing

### Local Testnet Management (`scripts/manage_local_testnet.sh`)

### Running Integration Tests (`scripts/run_tests_with_env.sh`)

## Migration

## Architecture Overview

MutAnt consists of five main components that work together to provide a complete storage solution:

1. **mutant-lib**: Core library handling chunking, encryption, and storage operations
2. **mutant-protocol**: Shared communication format definitions
3. **mutant-daemon**: Background service maintaining Autonomi connection
4. **mutant-client**: WebSocket client library for communicating with the daemon
5. **mutant-cli**: Command-line interface for end users

These components work together in a client-server architecture:

- The **daemon** uses **mutant-lib** to interact with the Autonomi network
- **Clients** (CLI or custom applications) connect to the daemon via WebSocket
- Communication between clients and the daemon uses the protocol definitions
- The daemon manages concurrent operations, background tasks, and network connections

This architecture provides several benefits:
- Persistent connection to the Autonomi network
- Background processing of long-running operations
- Task management and monitoring
- Concurrent operations with efficient resource usage
- Worker pools for optimized performance

### Internal Architecture
# Donate

Under the hood, MutAnt uses a worker architecture for handling operations:
- 10 workers, each with a dedicated client
- Each worker manages 10 concurrent operations (tasks)
- Total of 100 concurrent operations
- Round-robin distribution of work with work stealing
- Automatic recycling of failed pads
If you find this project useful, you can donate to support its development. <3

## Configuration
ETH/ANT `0x3376C33FdC0c885cef483690ffDd1e0Ff0Eb026c`
117 changes: 77 additions & 40 deletions mutant-cli/src/callbacks/put.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use super::progress::StyledProgressBar;
use colored::Colorize;
use indicatif::MultiProgress;
use log::{error, info, warn};
use mutant_client::ProgressReceiver;
Expand All @@ -13,6 +14,32 @@ struct PutCallbackContext {
confirm_pb_opt: Arc<Mutex<Option<StyledProgressBar>>>,
multi_progress: MultiProgress,
total_chunks: Arc<Mutex<usize>>,
first_complete_seen: Arc<Mutex<bool>>,
start_time: Arc<Mutex<std::time::Instant>>,
}

impl PutCallbackContext {
/// Finishes and clears a progress bar, setting it to 100% if not already finished.
/// Returns after the progress bar is cleared.
async fn finish_progress_bar(
&self,
pb_mutex: &Arc<Mutex<Option<StyledProgressBar>>>,
completion_message: &str,
bar_name: &str,
) {
let mut pb_guard = pb_mutex.lock().await;
if let Some(pb) = pb_guard.take() {
info!("Clearing {} bar", bar_name);
if !pb.is_finished() {
// If the bar isn't at 100%, set it to 100% before clearing
if let Some(len) = pb.length() {
pb.set_position(len);
}
pb.set_message(completion_message.to_string());
}
pb.finish_and_clear();
}
}
}

#[allow(clippy::type_complexity)]
Expand All @@ -21,13 +48,17 @@ pub fn create_put_progress(mut progress_rx: ProgressReceiver, multi_progress: Mu
let upload_pb_opt = Arc::new(Mutex::new(None::<StyledProgressBar>));
let confirm_pb_opt = Arc::new(Mutex::new(None::<StyledProgressBar>));
let total_chunks_arc = Arc::new(Mutex::new(0usize));
let first_complete_seen = Arc::new(Mutex::new(false));
let start_time = Arc::new(Mutex::new(std::time::Instant::now()));

let context = PutCallbackContext {
res_pb_opt: res_pb_opt.clone(),
upload_pb_opt: upload_pb_opt.clone(),
confirm_pb_opt: confirm_pb_opt.clone(),
multi_progress: multi_progress.clone(),
total_chunks: total_chunks_arc.clone(),
first_complete_seen: first_complete_seen.clone(),
start_time: start_time.clone(),
};

let ctx_clone = context.clone();
Expand Down Expand Up @@ -138,53 +169,59 @@ pub fn create_put_progress(mut progress_rx: ProgressReceiver, multi_progress: Mu
Ok::<bool, Box<dyn std::error::Error + Send + Sync>>(true)
}
PutEvent::Complete => {
info!("Complete event received, clearing progress bars");
// Check if this is the first or second Complete event
let mut first_complete_seen_guard = ctx.first_complete_seen.lock().await;
let is_first_complete = !*first_complete_seen_guard;

if is_first_complete {
info!("First Complete event received for data pads");
// Mark that we've seen the first Complete event
*first_complete_seen_guard = true;
drop(first_complete_seen_guard);

// Clear all progress bars before showing the message
// First, finish the reservation bar if it exists
ctx.finish_progress_bar(&ctx.res_pb_opt, "Pads acquired.", "reservation").await;

// Next, finish the upload bar
ctx.finish_progress_bar(&ctx.upload_pb_opt, "Upload complete.", "upload").await;

// Finally, finish the confirmation bar
ctx.finish_progress_bar(&ctx.confirm_pb_opt, "Confirmation complete.", "confirmation").await;

// Ensure all progress bars are cleared
crate::utils::ensure_progress_cleared(&ctx.multi_progress);

// Calculate elapsed time
let start_time = *ctx.start_time.lock().await;
let elapsed = start_time.elapsed();

// Format the elapsed time
let time_str = crate::utils::format_elapsed_time(elapsed);

// Display message about data pads being uploaded
let total_chunks = *ctx.total_chunks.lock().await;
println!("{} {} data pads have been uploaded (took {})",
"•".bright_green(),
total_chunks,
time_str);

return Ok::<bool, Box<dyn std::error::Error + Send + Sync>>(true);
}

// This is the second Complete event (or the only one for private keys)
info!("Final Complete event received, clearing progress bars");
drop(first_complete_seen_guard);

// Make sure all progress bars are finished and cleared
// First, finish the reservation bar if it exists
let mut res_pb_guard = ctx.res_pb_opt.lock().await;
if let Some(pb) = res_pb_guard.take() {
info!("Clearing reservation bar");
if !pb.is_finished() {
// If the bar isn't at 100%, set it to 100% before clearing
if let Some(len) = pb.length() {
pb.set_position(len);
}
pb.set_message("Pads acquired.".to_string());
}
pb.finish_and_clear();
}
drop(res_pb_guard);
ctx.finish_progress_bar(&ctx.res_pb_opt, "Pads acquired.", "reservation").await;

// Next, finish the upload bar
let mut upload_pb_guard = ctx.upload_pb_opt.lock().await;
if let Some(pb) = upload_pb_guard.take() {
info!("Clearing upload bar");
if !pb.is_finished() {
// If the bar isn't at 100%, set it to 100% before clearing
if let Some(len) = pb.length() {
pb.set_position(len);
}
pb.set_message("Upload complete.".to_string());
}
pb.finish_and_clear();
}
drop(upload_pb_guard);
ctx.finish_progress_bar(&ctx.upload_pb_opt, "Upload complete.", "upload").await;

// Finally, finish the confirmation bar
let mut confirm_pb_guard = ctx.confirm_pb_opt.lock().await;
if let Some(pb) = confirm_pb_guard.take() {
info!("Clearing confirmation bar");
if !pb.is_finished() {
// If the bar isn't at 100%, set it to 100% before clearing
if let Some(len) = pb.length() {
pb.set_position(len);
}
pb.set_message("Confirmation complete.".to_string());
}
pb.finish_and_clear();
}
drop(confirm_pb_guard);
ctx.finish_progress_bar(&ctx.confirm_pb_opt, "Confirmation complete.", "confirmation").await;

Ok::<bool, Box<dyn std::error::Error + Send + Sync>>(true)
}
Expand Down
2 changes: 1 addition & 1 deletion mutant-cli/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ pub enum Commands {
Rm { key: String },
#[command(about = "List stored keys")]
Ls {
#[arg(short = 'h', long, help = "Show fetch history")]
#[arg(long, help = "Show fetch history")]
history: bool,
},
#[command(about = "Show storage statistics")]
Expand Down
Loading
Loading