Skip to content

Conversation

@JoeZiminski
Copy link
Member

@JoeZiminski JoeZiminski commented Sep 11, 2025

This PR adds the option to encrypt the RClone config file containing SSH, AWS or GDrive connection information (e.g. private key, passwords, access tokens). The main changes are:

  1. SSH, AWS and GDrive RClone configs are now stored in separate .conf files (previously, the RClone default to store everything in RClone's rclone.conf was used. This is still kept for local_filesystem for backwards compatibility.
  2. There is now an option during set up connections (via Python API or TUI) to encrpy the config file. Doing so will encrypt the file using the operating systems credential manager.

This required quite a lot of refactoring to the setup connection functions (API and TUI) as well as the internal calls to rclone. Now, these calls are wrapped in some functionality that handles unencrypting the rclone config if required.

Tests and documentation have been updated.

This implementation is okay but there is still room for improvement. However, as this PR is pretty big and we just need to get this out, these are reversed for the future (#636).

@JoeZiminski JoeZiminski changed the title Refactor SSH connection to make adding passwords easier. Add password for SSH, AWS and GDrive config Sep 12, 2025
@JoeZiminski JoeZiminski marked this pull request as draft September 29, 2025 16:15
@JoeZiminski JoeZiminski force-pushed the add_password_to_rclone_config_for_aws_gdrive branch from 53a5a05 to 274fe29 Compare October 2, 2025 18:08
@JoeZiminski JoeZiminski mentioned this pull request Oct 7, 2025
@JoeZiminski JoeZiminski changed the base branch from main to refactor_ssh_connection October 7, 2025 21:46
@JoeZiminski JoeZiminski force-pushed the add_password_to_rclone_config_for_aws_gdrive branch 3 times, most recently from 44af5de to 5c21724 Compare November 1, 2025 01:25
Base automatically changed from refactor_ssh_connection to main November 3, 2025 15:30
@JoeZiminski JoeZiminski force-pushed the add_password_to_rclone_config_for_aws_gdrive branch from 350ac89 to 0c2c1b2 Compare November 3, 2025 15:43
@JoeZiminski JoeZiminski marked this pull request as ready for review November 3, 2025 21:39
Copy link
Collaborator

@cs7-shrey cs7-shrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @JoeZiminski, this is really well thought out. You have covered all the cases here very elegantly. I have dropped a few minor suggestions here and there, otherwise the implementation is logically and structurally very sound.

JoeZiminski and others added 23 commits December 19, 2025 17:15
Co-authored-by: Shrey Singh <96627769+cs7-shrey@users.noreply.github.com>
Co-authored-by: Shrey Singh <96627769+cs7-shrey@users.noreply.github.com>
….com:neuroinformatics-unit/datashuttle into add_password_to_rclone_config_for_aws_gdrive
…d encryption set up, and some small tidy ups.
Co-authored-by: Shrey Singh <96627769+cs7-shrey@users.noreply.github.com>
….com:neuroinformatics-unit/datashuttle into add_password_to_rclone_config_for_aws_gdrive
@JoeZiminski JoeZiminski added this to the v0.8.0 milestone Dec 20, 2025
Copy link
Collaborator

@cs7-shrey cs7-shrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just one small change.


self.interface = interface
self.no_browser_stage: None | str = "show_command_to_generate_code"
self.no_browser_stage: None | str = "pending"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant that this variable could be called something like self.client_secret_stage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants