Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: periodically garbage collect linux VHD blobs #5417

Draft
wants to merge 4 commits into
base: dev
Choose a base branch
from
Draft

chore: periodically garbage collect linux VHD blobs #5417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d0d4f32
chore: periodically garbage collect linux VHD blobs
Dec 12, 2024
cf92eda
chore: vars
Dec 12, 2024
4e2eb59
chore: pipeline vars
Dec 12, 2024
4ef6613
chore: naming
Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .pipelines/.vsts-garabge-collection.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,7 @@ jobs:
./vhdbuilder/scripts/gc.sh
env:
SUBSCRIPTION_ID: $(SUBSCRIPTION_ID)
VHD_BLOB_STORAGE_ACCOUNT_NAME: $(VHD_BLOB_STORAGE_ACCOUNT_NAME)
VHD_BLOB_STORAGE_CONTAINER_NAME: $(VHD_BLOB_STORAGE_CONTAINER_NAME)
DRY_RUN: ${{ parameters.DRY_RUN }}
displayName: Garbage collect resource groups
displayName: Garbage VHD build resources
20 changes: 20 additions & 0 deletions vhdbuilder/scripts/gc.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
set -euxo pipefail

[ -z "${SUBSCRIPTION_ID:-}" ] && echo "SUBSCRIPTION_ID must be set" && exit 1
[ -z "${VHD_BLOB_STORAGE_ACCOUNT_NAME}" ] && echo "VHD_BLOB_STORAGE_ACCOUNT_NAME must be set" && exit 1
[ -z "${VHD_BLOB_STORAGE_CONTAINER_NAME}" ] && echo "VHD_BLOB_STORAGE_CONTAINER_NAME must be set" && exit 1

SKIP_TAG_NAME="gc.skip"
SKIP_TAG_VALUE="true"
Expand All @@ -10,6 +12,7 @@ DRY_RUN="${DRY_RUN:-}"

DAY_AGO=$(( $(date +%s) - 86400 )) # 24 hours ago
WEEK_AGO=$(( $(date +%s) - 604800 )) # 7 days ago
WEEK_AGO_ISO=$(date @${WEEK_AGO} -Iseconds) # 7 days ago ISO format

function main() {
az login --identity # relies on an appropriately permissioned identity being attached to the build agent
Expand Down Expand Up @@ -47,6 +50,23 @@ function cleanup_rgs() {
done
}

function cleanup_storage_blobs() {
blobs=$(az storage blob list --account-name $VHD_BLOB_STORAGE_ACCOUNT_NAME --container-name $VHD_BLOB_STORAGE_CONTAINER_NAME --auth-mode login --query "[?properties.creationTime<='${WEEK_AGO_ISO}'].{name:name}" -o tsv || "")
for blob in $blobs; do
echo "will delete VHD blob $blob if unmodified since $WEEK_AGO_ISO..."
if [ "${DRY_RUN,,}" == "true" ]; then
echo "DRY_RUN: az storage blob delete --account-name $VHD_BLOB_STORAGE_ACCOUNT_NAME --container-name $VHD_BLOB_STORAGE_CONTAINER_NAME --name $blob --if-unmodified-since $WEEK_AGO_ISO --auth-mode login"
continue
fi
az storage blob delete \
--account-name $VHD_BLOB_STORAGE_ACCOUNT_NAME \
--container-name $VHD_BLOB_STORAGE_CONTAINER_NAME \
--name $blob \
--if-unmodified-since "$WEEK_AGO_ISO" \
--auth-mode login || return $?
done
}

function delete_group() {
local group=$1

Expand Down
Loading