Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
3093732
feat: s3 transfer manager v2
yenfryherrerafeliz Feb 2, 2025
5950c31
chore: add tests cases and refactor
yenfryherrerafeliz Feb 6, 2025
1c82ab5
chore: add multipart download listener tests
yenfryherrerafeliz Feb 6, 2025
237fc7b
chore: refactor multipart downloaders and add tests
yenfryherrerafeliz Feb 6, 2025
c28c165
feat: add download directory and refactor code
yenfryherrerafeliz Feb 13, 2025
34321b2
chore: add upload and refactor code
yenfryherrerafeliz Feb 14, 2025
5289f7c
feat: add upload directory feature
yenfryherrerafeliz Feb 14, 2025
c6c0780
feat: multipart upload and some refactor
yenfryherrerafeliz Feb 17, 2025
034b50d
chore: short namespace
yenfryherrerafeliz Feb 17, 2025
bc15ac3
chore: refactor and address feedback
yenfryherrerafeliz Feb 22, 2025
e681d10
chore: fix test cases
yenfryherrerafeliz Feb 24, 2025
1f094cd
chore: remove unused implementation
yenfryherrerafeliz Feb 24, 2025
464b498
chore: remove invalid test
yenfryherrerafeliz Feb 24, 2025
09e493f
fix: add nullable type
yenfryherrerafeliz Feb 24, 2025
f10522b
chore: add more tests
yenfryherrerafeliz Feb 26, 2025
a55e1b3
chore: add upload unit tests and refactor
yenfryherrerafeliz Mar 13, 2025
b271897
chore: address naming feedback and test failures
yenfryherrerafeliz Mar 13, 2025
f4f1c88
chore: address minor styling issues
yenfryherrerafeliz Mar 13, 2025
d987aff
chore: add download tests
yenfryherrerafeliz Mar 17, 2025
6d000f1
chore: add integ tests
yenfryherrerafeliz May 19, 2025
27570d0
chore: add integ test
yenfryherrerafeliz May 20, 2025
1dde7fc
chore: address PR feedback
yenfryherrerafeliz May 26, 2025
060e0e1
chore: fix and refactor
yenfryherrerafeliz May 27, 2025
ee0fefb
chore: fix TransferListener import
yenfryherrerafeliz May 27, 2025
9f639f1
chore: add test case
yenfryherrerafeliz May 29, 2025
bdce369
chore: address PR feedback
yenfryherrerafeliz Jun 4, 2025
cd2133a
fix: prevent calling twice downloadFailed
yenfryherrerafeliz Jun 5, 2025
0426e11
chore: fix exception throwing
yenfryherrerafeliz Jun 5, 2025
a548ce2
feat: consider checksum mode from command
yenfryherrerafeliz Jun 17, 2025
5a25ad8
chore: tests and minor fixes
yenfryherrerafeliz Jun 18, 2025
64fc3f6
tests: Add integ test for abort
yenfryherrerafeliz Jun 18, 2025
908b4a0
chore: update integ test
yenfryherrerafeliz Jun 18, 2025
1e643e5
feat: update to use modeled inputs
yenfryherrerafeliz Jul 10, 2025
48a2822
chore: multipart download updates
yenfryherrerafeliz Jul 15, 2025
02b43f0
chore: s3 transfer manager updates
yenfryherrerafeliz Jul 17, 2025
153227b
chore: minor update
yenfryherrerafeliz Jul 17, 2025
31dbd19
chore: add empty lines at the end
yenfryherrerafeliz Jul 17, 2025
aeed758
chore: remove unused implementations
yenfryherrerafeliz Jul 17, 2025
26d2469
fix: object key should be normalized
yenfryherrerafeliz Jul 17, 2025
10928ca
fix: minor logic and test fix
yenfryherrerafeliz Jul 18, 2025
632ece9
fix: fix s3 delimiter test
yenfryherrerafeliz Jul 18, 2025
6efcc0a
fix: wrong data provider name used
yenfryherrerafeliz Jul 18, 2025
b0c5f75
chore: addressed some styling
yenfryherrerafeliz Jul 18, 2025
44f6ff4
chore: update argument name
yenfryherrerafeliz Jul 18, 2025
83ccd9b
chore: make config optional
yenfryherrerafeliz Jul 22, 2025
039a67b
chore: minor refactor and fix
yenfryherrerafeliz Jul 23, 2025
50715d1
chore: make parameter optional
yenfryherrerafeliz Jul 23, 2025
3034ad8
chore: make model classes final
yenfryherrerafeliz Jul 28, 2025
cfe4ab5
chore: make classes final and refactor tests
yenfryherrerafeliz Jul 29, 2025
f4c42e0
chore: fix and reformat integ test
yenfryherrerafeliz Jul 30, 2025
9fb03f1
chore: address some styling suggestions
yenfryherrerafeliz Jul 31, 2025
4268266
chore: address PR suggestions
yenfryherrerafeliz Aug 4, 2025
59a8aa4
chore: null as default
yenfryherrerafeliz Aug 4, 2025
45ac289
chore: use UploadRequest construct
yenfryherrerafeliz Aug 11, 2025
20ada71
chore: address PR feedback
yenfryherrerafeliz Aug 18, 2025
e7842ed
chore: minor tests fix
yenfryherrerafeliz Aug 18, 2025
7c52a2f
enhancement: tests and fixes
yenfryherrerafeliz Aug 28, 2025
fa664b1
fix: tests with part count
yenfryherrerafeliz Aug 28, 2025
d2d4688
fix: test invalid checksum
yenfryherrerafeliz Aug 28, 2025
e6a141f
chore: address PR feedback
yenfryherrerafeliz Aug 28, 2025
e80eeff
chore: address some PR feedback
yenfryherrerafeliz Aug 29, 2025
ec383d2
chore: some minor refactor
yenfryherrerafeliz Aug 29, 2025
7fd10b6
enhancement: upload and download directory tests
yenfryherrerafeliz Oct 1, 2025
0c427ca
Merge branch 'master' into feat_s3_transfer_manager_v2
yenfryherrerafeliz Oct 2, 2025
b8a27c9
fix: fix setting checksum algorithm
yenfryherrerafeliz Oct 3, 2025
89d2c48
enhancement: add better handling for multipart uploads
yenfryherrerafeliz Oct 9, 2025
39d5e20
enhancement: add integ tests and minor fixes
yenfryherrerafeliz Oct 17, 2025
dd5f694
chore: address PR comments
yenfryherrerafeliz Nov 4, 2025
19f4436
chore: minor const reference fix
yenfryherrerafeliz Nov 4, 2025
4911c5e
feat: add concurrency support on downloads
yenfryherrerafeliz Nov 5, 2025
fe49aec
feat: resume multipart downloads
yenfryherrerafeliz Nov 12, 2025
d311f2f
fix: fix checksum calculation in download handler
yenfryherrerafeliz Nov 12, 2025
a04e43d
feat: resume uploads feature
yenfryherrerafeliz Nov 13, 2025
205d88f
fix: tests to work with new changes
yenfryherrerafeliz Nov 13, 2025
570a585
tests: add unit testing for resume operations
yenfryherrerafeliz Nov 13, 2025
ae47fef
tests: add integration tests for resume operations
yenfryherrerafeliz Nov 14, 2025
91c636b
chore: fix integ tests and make components final
yenfryherrerafeliz Nov 15, 2025
d9c5a54
chore: rename classes with abstract prefix
yenfryherrerafeliz Nov 15, 2025
d9d274f
chore: rename TransferRequest to AbstractTransferRequest
yenfryherrerafeliz Nov 15, 2025
6dde038
feat: add resume integ tests and minor fix
yenfryherrerafeliz Nov 15, 2025
3071ed2
chore: add rewind to the part body
yenfryherrerafeliz Nov 15, 2025
7230ed2
chore: address minor PR feedback.
yenfryherrerafeliz Nov 17, 2025
c24bdc0
chore: minor test adjustment
yenfryherrerafeliz Nov 19, 2025
e1a6a43
chore: remove default region
yenfryherrerafeliz Nov 19, 2025
bd03604
fix: default region in s3 transfer manager
yenfryherrerafeliz Nov 19, 2025
edb5496
chore: use trigger_error instead of error_log
yenfryherrerafeliz Nov 21, 2025
a763883
Merge branch 'feat_s3_transfer_manager_v2' into s3_transfer_manager_w…
yenfryherrerafeliz Nov 21, 2025
8e2f444
chore: fix merge issues
yenfryherrerafeliz Nov 21, 2025
a62bb2a
chore: merge branch 'master' into s3_transfer_manager_with_concurrent…
yenfryherrerafeliz Feb 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion features/s3Transfer/s3TransferManager.feature
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ Feature: S3 Transfer Manager
Examples:
| filename | content | checksum_algorithm |
| myfile-test-5-1.txt | This is a test file content #1 | crc32 |
| myfile-test-5-2.txt | This is a test file content #2 | crc32c |
| myfile-test-5-3.txt | This is a test file content #3 | sha256 |
| myfile-test-5-4.txt | This is a test file content #4 | sha1 |

Expand Down Expand Up @@ -139,4 +140,29 @@ Feature: S3 Transfer Manager
| file | size | algorithm | checksum |
| myfile-9-4 | 10485760 | crc32 | vMU7HA== |
| myfile-9-5 | 15728640 | crc32 | gjLQ1Q== |
| myfile-9-6 | 7340032 | crc32 | CKbfZQ== |
| myfile-9-6 | 7340032 | crc32 | CKbfZQ== |

Scenario Outline: Resume multipart download
Given I have a file <file> in S3 that requires multipart download
When I try the download for file <file>, with resume enabled, it fails
Then A resumable file for file <file> must exists
Then We resume the download for file <file> and it should succeed
Examples:
| file |
| resume-download-file-1.txt |
| resume-download-file-2.txt |
| resume-download-file-3.txt |
| resume-download-file-4.txt |

Scenario Outline: Resume multipart upload
Given I have a file <file> on disk that requires multipart upload
When I try to upload the file <file>, with resume enabled, it fails
Then A resumable file for file <file> must exists
Then We resume the upload for file <file> and it should succeed
Then The file <file> in s3 should match the local file
Examples:
| file |
| resume-upload-file-1.txt |
| resume-upload-file-2.txt |
| resume-upload-file-3.txt |
| resume-upload-file-4.txt |
21 changes: 20 additions & 1 deletion src/S3/CalculatesChecksumTrait.php
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

trait CalculatesChecksumTrait
{
private static $supportedAlgorithms = [
public static $supportedAlgorithms = [
'crc32c' => true,
'crc32' => true,
'sha256' => true,
Expand Down Expand Up @@ -56,4 +56,23 @@ public static function getEncodedValue($requestedAlgorithm, $value) {
. " Valid algorithms supported by the runtime are {$validAlgorithms}."
);
}

/**
* Returns the first checksum available in, if available.
*
* @param array $parameters
*
* @return string|null
*/
public static function filterChecksum(array $parameters):? string
{
foreach (self::$supportedAlgorithms as $algorithm => $_) {
$checksumAlgorithm = "Checksum" . strtoupper($algorithm);
if (isset($parameters[$checksumAlgorithm])) {
return $checksumAlgorithm;
}
}

return null;
}
}
Loading
Loading