Add multiple tasks at once.

[ryotarai]

Benchmark

$ head tmp/10000-tasks.yaml
payload: '1'
---
payload: '2'
---
payload: '3'
---
payload: '4'
---
payload: '5'
---

$ tail tmp/10000-tasks.yaml
---
payload: '9996'
---
payload: '9997'
---
payload: '9998'
---
payload: '9999'
---
payload: '10000'

$ cat tmp/benchmark.sh
set -x
make build-only
T=$(date +%s)
./dist/pftaskqueue create-queue benchmark-$T
./dist/pftaskqueue create-queue benchmark-6213725-$T
hyperfine "./dist/pftaskqueue add-task benchmark-$T -f tmp/10000-tasks.yaml" "./dist/pftaskqueue-6213725 add-task benchmark-6213725-$T -f tmp/10000-tasks.yaml"

+ make build-only
go build -tags netgo -installsuffix netgo -ldflags="-s -w -X github.com/pfnet-research/pftaskqueue/cmd.Version=dev -X github.com/pfnet-research/pftaskqueue/cmd.Revision=6213725 -extldflags \"-static\"" -o ./dist/pftaskqueue .
++ date +%s
+ T=1683682970
+ ./dist/pftaskqueue create-queue benchmark-1683682970
10:42AM INF Queue created successfully queueName=benchmark-1683682970 queueState=active queueUID=8f417fb9-593c-4e50-83b2-138bc14ad72d
+ ./dist/pftaskqueue create-queue benchmark-6213725-1683682970
10:42AM INF Queue created successfully queueName=benchmark-6213725-1683682970 queueState=active queueUID=2377b211-2d05-4a93-9f8e-789ea28e3bbe
+ hyperfine './dist/pftaskqueue add-task benchmark-1683682970 -f tmp/10000-tasks.yaml' './dist/pftaskqueue-6213725 add-task benchmark-6213725-1683682970 -f tmp/10000-tasks.yaml'
Benchmark 1: ./dist/pftaskqueue add-task benchmark-1683682970 -f tmp/10000-tasks.yaml
  Time (mean ± σ):     103.2 ms ±   4.9 ms    [User: 105.7 ms, System: 19.9 ms]
  Range (min … max):    97.0 ms … 118.2 ms    25 runs

Benchmark 2: ./dist/pftaskqueue-6213725 add-task benchmark-6213725-1683682970 -f tmp/10000-tasks.yaml
  Time (mean ± σ):      1.534 s ±  0.058 s    [User: 0.469 s, System: 0.493 s]
  Range (min … max):    1.445 s …  1.660 s    10 runs

Summary
  './dist/pftaskqueue add-task benchmark-1683682970 -f tmp/10000-tasks.yaml' ran
   14.86 ± 0.90 times faster than './dist/pftaskqueue-6213725 add-task benchmark-6213725-1683682970 -f tmp/10000-tasks.yaml'

[ryotarai]

新しいタスクをwatchする必要はないんじゃないかと思うので、消しています。もし間違ってそうだったら指摘いただけますと助かります。

[shioshiota]

LGTM

[shioshiota]

ここでvalidateTaskSpecできると、挙動が変わらず良いかと思ったんですが、backendのinterface変更が必要そうですし、このままで良いかも知れないですね。

[shioshiota]

@ryotarai
CI落ちてる部分確認お願いします 🙏

[shioshiota]

LGTM