Skip to content

Conversation

@shemogumbe
Copy link
Contributor

@shemogumbe shemogumbe commented Mar 25, 2024

Closes #112

What does this PR do

Enables uploading large files using the python SDK, as chunks of bytes

Demo

Upload_chuknks2

Usage and Testing Instructions

destination_path = "path/to/your_file.txt"
file_path = "path/to/your_file.txt"


async def upload_large_file():
    try:
        file = open(file_path, 'rb')
        uploadable_properties = DriveItemUploadableProperties(
            additional_data={'@microsoft.graph.conflictBehavior': 'replace'}
        )
        upload_session_request_body = CreateUploadSessionPostRequestBody(item=uploadable_properties)
        print(f"Uploadable Properties: {uploadable_properties.additional_data}")
        # can be used for normal drive uploads
        try:

            upload_session = await user_client.drives.by_drive_id(
                "b!WtUPKhiPm0a6Bj6Z_J97a53XIv_KoNNAkdLRUrSF06lh-qW6JpABSoW62oPNb03R"
            ).items.by_drive_item_id('root:/my_docs/test_upload.txt:'
                                     ).create_upload_session.post(upload_session_request_body)
            
        except APIError as ex:
            print(f"Error creating upload session: {ex}")

        # to be used for large file uploads
        large_file_upload_session = LargeFileUploadSession(
            upload_url=upload_session.upload_url,
            expiration_date_time=datetime.now() + timedelta(days=1),
            additional_data=upload_session.additional_data,
            is_cancelled=False,
            next_expected_ranges=upload_session.next_expected_ranges
        )

        # max_slice_size = 320 * 1024 - remove to use default
        task = LargeFileUploadTask(large_file_upload_session, user_client.request_adapter, file)
        total_length = os.path.getsize(file_path)
        
        # Upload the file
        # The callback
        def progress_callback(uploaded_byte_range: tuple[int, int]):
            print(f"Uploaded {uploaded_byte_range[0]} bytes of {total_length} bytes\n\n")

        try:
            upload_result = await task.upload(progress_callback)
            print(f"Upload complete {upload_result}")
        except APIError as ex:
            print(f"Error uploading: {ex.message} - {ex.response_status_code}")
    except APIError as e:
        print(f"Error: {e}")


asyncio.run(upload_large_file())
  • Install msgraphcore and msgraph-sdk
    *Create a script with appropriate authentication and scopes
  • Run the code above, with appropriate filepath and destination address
  • Check that your logs are as per the screenshot above
  • Visit the destination folder in ondedrive and counter check that the file is there, you can us echecksum to see if the files are identical

@github-actions
Copy link
Contributor

github-actions bot commented Apr 8, 2024

This pull request has conflicting changes, the author must resolve the conflicts before this pull request can be merged.

@github-actions
Copy link
Contributor

Conflicts have been resolved. A maintainer will take a look shortly.

@shemogumbe shemogumbe self-assigned this Apr 20, 2024
@shemogumbe shemogumbe marked this pull request as ready for review April 22, 2024 14:08
Copy link
Contributor

@Ndiritu Ndiritu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Some unit tests would also be helpful for the completion scenarios on Outlook vs OneDrive/Sharepoint

@shemogumbe shemogumbe requested a review from Ndiritu April 29, 2024 14:24
@shemogumbe shemogumbe force-pushed the shem/large_file_upload branch from e751b49 to 9754d1c Compare April 29, 2024 14:53
@shemogumbe shemogumbe merged commit eb0fd78 into main May 2, 2024
@shemogumbe shemogumbe deleted the shem/large_file_upload branch May 2, 2024 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Developers should be able to access file upload support on the Python SDK

4 participants