Skip to content

MS-1058 Implementation of signed url sample upload#1255

Merged
luhmirin-s merged 9 commits into
mainfrom
feature/MS-1058-sample-upload-2
Jul 16, 2025
Merged

MS-1058 Implementation of signed url sample upload#1255
luhmirin-s merged 9 commits into
mainfrom
feature/MS-1058-sample-upload-2

Conversation

@luhmirin-s
Copy link
Copy Markdown
Contributor

@luhmirin-s luhmirin-s commented Jul 10, 2025

JIRA ticket
Will be released in: 2025.3.0

Notable changes

  • Changed the image repository interface to accommodate for the extra information required for sample upload with signed URLs.
  • Moved sample file path creation into the module to ensure consistent paths and have an option to break that path back into usable metadata about the sample (session ID, modality and sample ID).
  • Refactored the existing Firestore image uploader out of the ImageRepositoryImpl to allow for different approaches to reading files from the disk:
    • Firestore - reads and uploads files one by one in sequence until all are uploaded.
    • Signed URL - reads files in batches to calculate size and MD5 values, requests the upload URLs for the whole batch with a request, then proceeds to upload files one by one within the batch until all batches are uploaded.
  • Important - encrypted file input stream that we are getting from the local disc can only be read once and cannot be reset, therefore any BODY-level network logging (both chucker and okhttp) will consume the stream to print it out, and the actual file upload will fail.

Testing guidance

  • Enable sample/image upload in Vulcan.
  • Do several biometric flows with face or fingerprint tracking sessions IDs. Wait a couple of hours until the state in sync info screen shows that images were uploaded. Check Firestore for the uploaded images by project and session IDs.
  • Enable signed URL upload with custom configuration.
  • Do several biometric flows with face or fingerprint tracking sessions IDs. Wait a couple of hours until the state in sync info screen shows that images were uploaded. Check the new sample storage for the uploaded images by project and session IDs.

Additional work checklist

  • Effect on other features and security has been considered
  • Design document marked as "In development" (if applicable)
  • External (Gitbook) and internal (Confluence) Documentation is up to date (or ticket created)
  • Test cases in Testiny are up to date (or ticket created)
  • Other teams notified about the changes (if applicable)

@cla-bot cla-bot Bot added the ... label Jul 10, 2025
@luhmirin-s luhmirin-s marked this pull request as ready for review July 14, 2025 13:02
@luhmirin-s luhmirin-s requested review from a team, BurningAXE, TristramN, alex-vt, alexandr-simprints, meladRaouf and ybourgery and removed request for a team July 14, 2025 13:03
@luhmirin-s luhmirin-s changed the title MS-1058 Implementation of signed url sample upload (WIP) MS-1058 Implementation of signed url sample upload Jul 14, 2025
@luhmirin-s
Copy link
Copy Markdown
Contributor Author

@alexandr-simprints I have added more general logging around the upload to help us restore the timeline in crash logs (if it will be necessary). There isn't much else we could add to those logs, tho.

@luhmirin-s luhmirin-s force-pushed the feature/MS-1058-sample-upload-2 branch from 7f579f1 to 3232996 Compare July 15, 2025 06:05
@sonarqubecloud
Copy link
Copy Markdown

@luhmirin-s luhmirin-s merged commit fa8391d into main Jul 16, 2025
12 checks passed
@luhmirin-s luhmirin-s deleted the feature/MS-1058-sample-upload-2 branch July 16, 2025 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants