Get status of bulk request

[hailihu]

See #86

The command to follow status request is:

ada --stat-request <request-id> ...

Also, changed output of (un)stage request, to help user understand what to do with request URL and ID

[onnozweers]

Nice! Seems to work well.

One remark: when literally copying and pasting the suggested ada --stat-request command, I get errors.

[onno@mylaptop ~/git/Ada] % ada/ada --netrc --stage /users/onno/tape/test
Loading ~/.ada/ada.conf ...
Information about bulk request is logged in /Users/zweer001/.ada/requests.log
To check status of request, paste request URL in browser:
   request-url: https://dcachetest.grid.surfsara.nl:20443/api/v1/bulk-requests/d8e23d99-9780-49ac-a52f-8dedfa966436
Or use command:
   ada --stat-request d8e23d99-9780-49ac-a52f-8dedfa966436

[onno@mylaptop ~/git/Ada] % ada --stat-request d8e23d99-9780-49ac-a52f-8dedfa966436   
zsh: command not found: ada

When I fix that, the next hurdle is that the authentication doesn't match:

[onno@mylaptop ~/git/Ada] % ada/ada --stat-request d8e23d99-9780-49ac-a52f-8dedfa966436
Loading ~/.ada/ada.conf ...
tr: Illegal byte sequence
Token source: $BEARER_TOKEN
ERROR: Token has expired 39726 seconds ago.

For me no problem because I know what to do. But I expect there will be some users who may struggle with this.

Could we make the printed command resemble the used command a bit more, for the executable ($0) and the authentication method (--proxy, --netrc, --tokenfile etc)?

[onnozweers]

I notice that the status information returned by dCache does not show everything. In this example it doesn't show that the command has been a "--stage" or pin. If we think users need that, I could submit a request to the dCache developers for that.

{
  "nextId" : -1,
  "uid" : "d8e23d99-9780-49ac-a52f-8dedfa966436",
  "arrivedAt" : 1759845729900,
  "startedAt" : 1759845729914,
  "lastModified" : 1759846024436,
  "status" : "COMPLETED",
  "targetPrefix" : "/",
  "targets" : [ {
    "target" : "/users/onno/tape/test",
    "state" : "COMPLETED",
    "submittedAt" : 1759845729905,
    "startedAt" : 1759845729905,
    "finishedAt" : 1759846024432,
    "id" : 35771
  } ]
}

[hailihu]

Nice! Seems to work well.

One remark: when literally copying and pasting the suggested ada --stat-request command, I get errors.

[onno@mylaptop ~/git/Ada] % ada/ada --netrc --stage /users/onno/tape/test
Loading ~/.ada/ada.conf ...
Information about bulk request is logged in /Users/zweer001/.ada/requests.log
To check status of request, paste request URL in browser:
   request-url: https://dcachetest.grid.surfsara.nl:20443/api/v1/bulk-requests/d8e23d99-9780-49ac-a52f-8dedfa966436
Or use command:
   ada --stat-request d8e23d99-9780-49ac-a52f-8dedfa966436

[onno@mylaptop ~/git/Ada] % ada --stat-request d8e23d99-9780-49ac-a52f-8dedfa966436   
zsh: command not found: ada

When I fix that, the next hurdle is that the authentication doesn't match:

[onno@mylaptop ~/git/Ada] % ada/ada --stat-request d8e23d99-9780-49ac-a52f-8dedfa966436
Loading ~/.ada/ada.conf ...
tr: Illegal byte sequence
Token source: $BEARER_TOKEN
ERROR: Token has expired 39726 seconds ago.

For me no problem because I know what to do. But I expect there will be some users who may struggle with this.

Could we make the printed command resemble the used command a bit more, for the executable ($0) and the authentication method (--proxy, --netrc, --tokenfile etc)?

Done, can you please check if you agree with the implementation? The output given can now be directly copied and executed.

[hailihu]

I notice that the status information returned by dCache does not show everything. In this example it doesn't show that the command has been a "--stage" or pin. If we think users need that, I could submit a request to the dCache developers for that.

{
  "nextId" : -1,
  "uid" : "d8e23d99-9780-49ac-a52f-8dedfa966436",
  "arrivedAt" : 1759845729900,
  "startedAt" : 1759845729914,
  "lastModified" : 1759846024436,
  "status" : "COMPLETED",
  "targetPrefix" : "/",
  "targets" : [ {
    "target" : "/users/onno/tape/test",
    "state" : "COMPLETED",
    "submittedAt" : 1759845729905,
    "startedAt" : 1759845729905,
    "finishedAt" : 1759846024432,
    "id" : 35771
  } ]
}

It may be handy, but not essential as we also store this information in the requests.log

[onnozweers]

Doesn't seem to work with --netrc

(git branch: 86-get-status-request)
[onno@mylaptop ~/git/Ada] % ada/ada --netrc --stage /users/onno/tape/test-badger14-10
Information about bulk request is logged in /Users/zweer001/.ada/requests.log
To check status of request, paste request URL in browser:
   request-url: https://dcachetest.grid.surfsara.nl:20443/api/v1/bulk-requests/1c321bc5-344f-4e59-bb6c-f5985a1d8472
Or use command:
   ada/ada --stat-request 1c321bc5-344f-4e59-bb6c-f5985a1d8472 --api https://dcachetest.grid.surfsara.nl:20443/api/v1

(git branch: 86-get-status-request)
[onno@mylaptop ~/git/Ada] % ada/ada --stat-request 1c321bc5-344f-4e59-bb6c-f5985a1d8472 --api https://dcachetest.grid.surfsara.nl:20443/api/v1
ERROR: you have to specify a valid authentication method.

Shouldn't we look closer to the original command line arguments? If the ada --stage command didn't have --api, we probably won't need it in the ada --stat-request command either.

[onnozweers]

Would it be an idea to make the timestamps human readable? A jq like this could do that.

jq ' 
  walk(
    if type == "object" then
      with_entries(
        if (.key|test("(At$|Modified$)")) and (.value|type=="number") then
          .value |= (. / 1000 | strflocaltime("%F %T"))
        else
          .
        end
      )
    else
      .
    end
  )
'

[onnozweers]

Oops, in the last commit I wrote "stderr instead of stdin" but of course I meant "stderr instead of stdout". 😅

[hailihu]

Doesn't seem to work with --netrc

(git branch: 86-get-status-request)
[onno@mylaptop ~/git/Ada] % ada/ada --netrc --stage /users/onno/tape/test-badger14-10
Information about bulk request is logged in /Users/zweer001/.ada/requests.log
To check status of request, paste request URL in browser:
   request-url: https://dcachetest.grid.surfsara.nl:20443/api/v1/bulk-requests/1c321bc5-344f-4e59-bb6c-f5985a1d8472
Or use command:
   ada/ada --stat-request 1c321bc5-344f-4e59-bb6c-f5985a1d8472 --api https://dcachetest.grid.surfsara.nl:20443/api/v1

(git branch: 86-get-status-request)
[onno@mylaptop ~/git/Ada] % ada/ada --stat-request 1c321bc5-344f-4e59-bb6c-f5985a1d8472 --api https://dcachetest.grid.surfsara.nl:20443/api/v1
ERROR: you have to specify a valid authentication method.

Shouldn't we look closer to the original command line arguments? If the ada --stage command didn't have --api, we probably won't need it in the ada --stat-request command either.

I was thinking in this direction first, but that means we have to go through all the arguments and pick out the ones we need. It sounds simple, but since the way arguments can be passed is very flexible (with or without optional arguments), I didn't see an elegant way to do this (without adding some lines code). I'll have another look.

[hailihu]

Would it be an idea to make the timestamps human readable? A jq like this could do that.

jq ' 
  walk(
    if type == "object" then
      with_entries(
        if (.key|test("(At$|Modified$)")) and (.value|type=="number") then
          .value |= (. / 1000 | strflocaltime("%F %T"))
        else
          .
        end
      )
    else
      .
    end
  )
'

I don't know, I think it's more elegant to return directly what dCache API returns. Perhaps this json output changes in the furutre, or users want to do something with it.

[onnozweers]

Looks good! Let's merge!