Skip to content

Feat/noninteractive shell e2e#1845

Merged
KCarretto merged 17 commits intomainfrom
feat/noninteractive-shell-e2e
Feb 22, 2026
Merged

Feat/noninteractive shell e2e#1845
KCarretto merged 17 commits intomainfrom
feat/noninteractive-shell-e2e

Conversation

@KCarretto
Copy link
Copy Markdown
Collaborator

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

KCarretto and others added 6 commits February 21, 2026 15:36
* Implement agent-side shell features and synchronization

- Update `ShellTask` schema with execution timestamps.
- Update `c2` and `portal` protocol buffers to support `ShellTask` and `ShellTaskOutput`.
- Implement `ClaimTasks` and `ReportTaskOutput` logic for shell tasks in Go backend.
- Implement `ShellManager` in Rust agent (`imix`) to manage `Eldritch` interpreters per shell.
- Integrate `ShellManager` with `ImixAgent` and `portal` streams for input/output routing.
- Ensure shell tasks are claimed and output is buffered/reported correctly.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Implement agent-side shell features and synchronization

- Update `ShellTask` schema with execution timestamps.
- Update `c2` and `portal` protocol buffers to support `ShellTask` and `ShellTaskOutput`.
- Implement `ClaimTasks` and `ReportTaskOutput` logic for shell tasks in Go backend.
- Implement `ShellManager` in Rust agent (`imix`) to manage `Eldritch` interpreters per shell.
- Integrate `ShellManager` with `ImixAgent` and `portal` streams for input/output routing.
- Ensure shell tasks are claimed and output is buffered/reported correctly.
- Fixed `tavern/internal/http/shell/handler.go` by removing invalid `StreamId` initialization.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Fix ShellPayload compilation error in tavern

- Removed invalid `StreamId` field initialization from `portalpb.ShellPayload` struct literal in `tavern/internal/http/shell/handler.go`.
- This field was removed from the protobuf definition in a previous commit.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Fix Rust compilation errors in implants/imix and stdlib

- Added missing `shell_task_output: None` to `ReportTaskOutputRequest` struct literals in `implants/imix/src/tests/agent_trait_tests.rs` and `implants/lib/eldritch/stdlib/eldritch-libagent/src/std/report_task_output_impl.rs` to match the updated protobuf definition.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Run cargo fmt to ensure consistent code style

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Fix ShellPrinter output sequence ID handling

- Updated `c2.proto` to include `sequence_id` and `stream_id` in `ShellTask`.
- Updated `tavern` to populate these fields when claiming tasks.
- Updated `imix` `ShellManager` to propagate `sequence_id` and `stream_id` from input tasks/motes to output motes.
- Fixed an issue where output was being dropped in interactive sessions due to mismatched sequence IDs (output motes were using `seq_id: 0`).

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>
)

Updated `implants/imix/src/agent.rs` to aggregate `ReportTaskOutputRequest` messages based on their content type:
- Standard task outputs (`output` field) are aggregated by `task_id`.
- Shell task outputs (`shell_task_output` field) are aggregated by `shell_task_id`.

This ensures that shell task outputs are correctly reported and not merged with standard task outputs (or dropped/overwritten if they share the default task context).

Also added a new test `test_agent_output_aggregation` in `implants/imix/src/tests/agent_output_aggregation.rs` to verify this behavior.
Fixed a missing field `shell_tasks` in `ClaimTasksResponse` construction in `implants/imix/src/tests/task_tests.rs`.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>
…erpret() (#1843)

Refactored `implants/imix/src/shell/manager.rs` to:
- Capture the result of `state.interpreter.interpret(&task.input)`.
- Use helper functions `send_shell_output` and `send_shell_error` to send `ReportTaskOutputRequest` and `Mote` messages.
- Send the `Debug` representation of the result (if not `Value::None`) as task output, consistent with REPL behavior.
- Ensure correct `Arc` cloning to avoid borrow checker errors.
- Handle interpretation errors by reporting them as task errors.

This ensures that return values of shell commands (e.g. `process.list()`) are displayed to the user even if they are not explicitly printed.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>
Moved `StreamPrinter` from `imix/src/task.rs` to `imix/src/printer.rs` and updated it to distinguish between `Stdout` and `Stderr` using an `OutputKind` enum.
Updated `imix/src/shell/repl.rs` to reuse `StreamPrinter` instead of the custom `ShellPrinter`.
Refactored `imix/src/task.rs` to use the shared `StreamPrinter` and properly report stderr to `TaskError` field.
Updated `imix/src/lib.rs` and `imix/src/main.rs` to include the new `printer` module.
This change unifies output handling across the agent and improves maintainability.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 22, 2026

Summary

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Other ❓ Flaky 🍂 Duration ⏱️
2624    ±0 2624    ±0 0    ±0 0    ±0 0    ±0 0    ±0 1ms    ±0

Previous Results

Build 🏗️ Result 🧪 Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Other ❓ Flaky 🍂 Duration ⏱️
#1072 2624 2624 0 0 0 0 37.0s

Insights

Average Tests per Run Total Flaky Tests Total Failed Slowest Test (p95)
2624 0 0 5.2s

Slowest Tests

Test 📝 Results 📊 Duration (avg) ⏱️ Duration (p95) ⏱️
eldritch-libsys: std::dll_inject_impl::tests::test_dll_inject_simple 1 5.2s 5.2s
TestDockerExecutor_Build_ContextCancellation 1 5.2s 5.2s
TestInteractiveShell 1 5.1s 5.1s
TestOtherStreamOutput 1 5.0s 5.0s
imix::bin/imix: install::tests::test_install_execution 3 1.5s 4.3s
imix::bin/imix: install::tests::test_install_execution 3 1.5s 4.3s
imix::bin/imix: install::tests::test_install_execution 3 1.5s 4.3s
imix::bin/imix: tests::task_tests::test_task_streaming_error 3 3.0s 3.0s
imix::bin/imix: tests::task_tests::test_task_streaming_error 3 3.0s 3.0s
imix::bin/imix: tests::task_tests::test_task_streaming_error 3 3.0s 3.0s

🎉 No failed tests in this run. | 🍂 No flaky tests in this run.

Github Test Reporter by CTRF 💚

🔄 This comment has been updated

KCarretto and others added 10 commits February 21, 2026 23:37
- Modified `StreamPrinter` in `implants/imix/src/printer.rs` to use separate channels for stdout and stderr.
- Updated `implants/imix/src/task.rs` to instantiate `StreamPrinter` with two channels and consume them concurrently using `tokio::select!`.
- Updated `implants/imix/src/shell/repl.rs` to consume both stdout and stderr streams in the REPL loop.
- Verified that `eprint` output is correctly routed to the `TaskError` field via existing tests.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>
* Refactor imix shell manager to run interpreter in blocking task

Fixes a panic "Cannot start a runtime from within a runtime" when the eldritch
interpreter calls async agent functions (which use `block_on`) from within
the imix shell manager's async runtime context.

Refactored `ShellManager::handle_task` and `handle_portal_payload` to offload
the synchronous `interpreter.interpret` call to `tokio::task::spawn_blocking`.
Introduced `run_interpreter_task` helper to manage state transfer between
async and blocking contexts.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Refactor imix shell manager to run interpreter in dedicated blocking thread

Refactored `ShellManager` to execute the `eldritch` interpreter in a dedicated, long-running blocking thread (`tokio::task::spawn_blocking`) for each shell session.

This solves a panic ("Cannot start a runtime from within a runtime") caused by the interpreter calling `block_on` from within an async runtime context.

Introduced `InterpreterCommand` enum and `InterpreterHandle` to manage communication between the async manager and the blocking interpreter thread via `tokio::sync::mpsc` channels. This ensures tasks are processed sequentially and prevents state race conditions.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>
* Refactor shell_manager_tx in portals to use mpsc::Sender directly

This change simplifies the passing of `shell_manager_tx` to portal functions by removing the `Arc<Mutex<Option<...>>>` wrapper. The sender is now passed directly as `mpsc::Sender<ShellManagerMessage>`.
- Updated `run_create_portal` in `implants/imix/src/portal/mod.rs` to accept `mpsc::Sender`.
- Updated `run` and `handle_incoming_mote` in `implants/imix/src/portal/run.rs` to accept `mpsc::Sender` and remove unnecessary locking.
- Updated `create_portal` in `implants/imix/src/agent.rs` to extract and clone the sender before spawning the portal task.
- `handle_incoming_mote` now uses `try_send` to avoid blocking the portal loop on shell manager backpressure.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Refactor shell_manager_tx in ImixAgent to be a direct mpsc::Sender

- Updated `ImixAgent` struct: `shell_manager_tx` is now `tokio::sync::mpsc::Sender<ShellManagerMessage>`.
- Introduced `shell_manager_rx: Arc<Mutex<Option<...>>>` to handle the receiver side in `ImixAgent::new`.
- Updated `start_shell_manager` to claim the receiver from `shell_manager_rx`.
- Simplified usages of `shell_manager_tx` in `process_job_request` and `create_portal` (removed unnecessary locking/cloning).
- This aligns with the requirement that `shell_manager_tx` should not be an Option and should be readily available.

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

* Refactor shell_manager_tx in ImixAgent to be a direct mpsc::Sender

- Updated `ImixAgent` struct: `shell_manager_tx` is now `tokio::sync::mpsc::Sender<ShellManagerMessage>`.
- Updated `ImixAgent::new` to accept `shell_manager_tx` as an argument, implementing dependency injection.
- Updated `start_shell_manager` to accept a `ShellManager` instance instead of creating the channel internally.
- Refactored `run_agent` in `run.rs` to create the channel and `ShellManager` explicitly.
- Updated all relevant tests to match the new API.
- Simplified usages of `shell_manager_tx` in `process_job_request` and `create_portal` (removed unnecessary locking/cloning/unwrapping).

Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: KCarretto <16250309+KCarretto@users.noreply.github.com>
@KCarretto KCarretto requested a review from hulto February 22, 2026 19:14
@KCarretto KCarretto marked this pull request as ready for review February 22, 2026 19:15
Copy link
Copy Markdown
Collaborator

@hulto hulto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Is there a new shell task ent? If so does it need to get added to dropalldata?

@KCarretto KCarretto added this pull request to the merge queue Feb 22, 2026
Merged via the queue into main with commit eecb4ab Feb 22, 2026
8 checks passed
@KCarretto KCarretto deleted the feat/noninteractive-shell-e2e branch February 22, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants