Skip to content

Commit

Permalink
chore(IDX): in system-tests, show error when uploading the image fail…
Browse files Browse the repository at this point in the history
…s & retry up to 3 times (#2954)

In system-tests, the image upload sometimes fails. For example:
```
$ ict test //rs/tests/consensus/subnet_recovery:sr_app_same_nodes_test
Raw Bazel command to be invoked: 
$ bazel test //rs/tests/consensus/subnet_recovery:sr_app_same_nodes_test --config=systest --cache_test_results=no
INFO: Invocation ID: 075ad521-6af0-4c9b-a0d4-37b0024ff087
INFO: Streaming build results to: https://dash.idx.dfinity.network/invocation/075ad521-6af0-4c9b-a0d4-37b0024ff087
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
INFO: Analyzed target //rs/tests/consensus/subnet_recovery:sr_app_same_nodes_test (0 packages loaded, 0 targets configured).
Found dep to upload ic-os/guestos/envs/dev/disk-img.tar.zst (5b62b9686a6007ac1adfea3436ee588fccc3dfde35c0f9d20dd476ea7cd7f07b)
dep 'ic-os/guestos/envs/dev/disk-img.tar.zst': not uploaded yet
Using upload URL: 'http://server.bazel-remote.svc.cluster.local:8080/cas/5b62b9686a6007ac1adfea3436ee588fccc3dfde35c0f9d20dd476ea7cd7f07b'
0 0.015513 0
FAIL: //rs/tests/consensus/subnet_recovery:sr_app_same_nodes_test 
```

This is confusing for users since no curl error is shown. This commit
fixes that by passing `--show-error` to curl and also retrying the
upload up to 3 times to make it more robust to temporary failures.
  • Loading branch information
basvandijk authored Dec 3, 2024
1 parent 05f6602 commit 8b74dc5
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion bazel/upload_systest_dep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ else
dep_upload_url="$UPLOAD_URL/$dep_sha256"
echo "Using upload URL: '$dep_upload_url'" >&2
curl_out=$(mktemp)
curl --silent --fail "$dep_upload_url" --upload-file "$dep_filename" -w '%{size_upload} %{time_total} %{speed_upload}\n' | tee "$curl_out" >&2
curl --silent --show-error --fail --retry 3 "$dep_upload_url" --upload-file "$dep_filename" -w '%{size_upload} %{time_total} %{speed_upload}\n' | tee "$curl_out" >&2
# read & pretty print 3 metrics: upload size, upload time & upload speed
if read -ra metrics <"$curl_out"; then
echo "Uploaded $(numfmt --to=iec-i --suffix=B "${metrics[0]}") in ${metrics[1]}s ($(numfmt --to=iec-i --suffix=B "${metrics[2]}")/s)" >&2
Expand Down

0 comments on commit 8b74dc5

Please sign in to comment.