Skip to content

GHAC multi-part writes only take the last part, causing commit errors #3162

@huonw

Description

@huonw

Thanks for OpenDAL!

It seems that the GitHub Actions Cache backend doesn't work for multi-part writes (e.g. using Operator::writer(...), and only the last part is applied. This leads to errors during close, when the "commit" call sends the wrong size (the real size != the size of the last part), like the following:

Unexpected (permanent) at Writer::close => {"$id":"1","innerException":null,"message":"The cache content was partially missing or did not match the expected size.","typeName":"Microsoft.Azure.DevOps.ArtifactCache.WebApi.InvalidSealRequestException, Microsoft.Azure.DevOps.ArtifactCache.WebApi","typeKey":"InvalidSealRequestException","errorCode":0,"eventId":3000}

Example:

use tokio::io::AsyncWriteExt;

#[tokio::main]
async fn main() {
  let service = opendal::services::Ghac::default();
  let op = opendal::Operator::new(service).unwrap().finish();

  // unique for every run to test behaviour:
  let now = std::time::SystemTime::now().duration_since(std::time::SystemTime::UNIX_EPOCH).unwrap();

  let key = format!("key-{}", now.as_secs_f64());
  let mut writer = op.writer(&key).await.unwrap();

  writer.write_all(&[0]).await.unwrap();
  writer.write_all(&[1]).await.unwrap();
  writer.close().await.unwrap();
}

This crashes with an error like the above:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Unexpected (permanent) at Writer::close => {"$id":"1","innerException":null,"message":"The cache content was partially missing or did not match the expected size.","typeName":"Microsoft.Azure.DevOps.ArtifactCache.WebApi.InvalidSealRequestException, Microsoft.Azure.DevOps.ArtifactCache.WebApi","typeKey":"InvalidSealRequestException","errorCode":0,"eventId":3000}

Context:
   uri: https://acghubeus1.actions.githubusercontent.com/swYrhypG2MJmMQxwjZTKUlTTHNgLiRmyMXAqEruwvhFVEC0MZe/_apis/artifactcache/caches/3861
   response: Parts { status: 400, version: HTTP/2.0, headers: {"content-type": "application/json; charset=utf-8", "date": "Sat, 23 Sep 2023 03:36:46 GMT", "server": "Kestrel", "cache-control": "no-store,no-cache", "pragma": "no-cache", "content-length": "320", "strict-transport-security": "max-age=2592000", "x-tfs-processid": "c7cabc2a-d1d3-49ee-a0b3-228b04d2fe0e", "activityid": "29909b20-8a26-485d-9e4d-bb787b99ace0", "x-tfs-session": "29909b20-8a26-485d-9e4d-bb787b99ace0", "x-vss-e2eid": "29909b20-8a26-485d-9e4d-bb787b99ace0", "x-vss-senderdeploymentid": "e3de3c8c-fa12-318e-3301-e1d3e326b2e8", "x-frame-options": "SAMEORIGIN"} }
   called: Backend::ghac_commit
   service: ghac
   path: key-1695440205.717675

Running against https://github.com/nektos/act (which doesn't enforce the commit size) reveals that only the &[1] is stored.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions