Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add snapshot tests for internal Dockerfiles #6142

Open
wants to merge 5 commits into
base: nightly
Choose a base branch
from
Open

Add snapshot tests for internal Dockerfiles #6142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0ec0d4b
Add basic generated artifacts snapshot test
lbussell Jan 8, 2025
4e3fc3a
Use scrubbed internal Dockerfiles
lbussell Jan 9, 2025
783996b
Accept baselines
lbussell Jan 9, 2025
9e3bafa
Update gitattributes
lbussell Jan 9, 2025
df2ed7a
Add dotnet tool configuration
lbussell Jan 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
13 changes: 13 additions & 0 deletions .config/dotnet-tools.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"version": 1,
"isRoot": true,
"tools": {
"verify.tool": {
"version": "0.6.0",
"commands": [
"dotnet-verify"
],
"rollForward": false
}
}
}
2 changes: 2 additions & 0 deletions .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,5 @@ src/coreclr/tests/src/JIT/Performance/CodeQuality/BenchmarksGame/reverse-complem
src/coreclr/tests/src/JIT/Performance/CodeQuality/BenchmarksGame/k-nucleotide/knucleotide-input.txt text eol=lf
src/coreclr/tests/src/JIT/Performance/CodeQuality/BenchmarksGame/k-nucleotide/knucleotide-input-big.txt text eol=lf
src/coreclr/tests/src/performance/Scenario/JitBench/Resources/word2vecnet.patch text eol=lf

tests/Microsoft.DotNet.Docker.Tests/Baselines/** linguist-generated
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@ artifacts/

# ImageBuilder directory
.Microsoft.DotNet.ImageBuilder

# Verify.Xunit library
*.received.*
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/Dockerfile.linux.download-file
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
sha-var-name: Name of variable that stores the checksum
sha-function: SHA function to use ^

set isInternal to find(ARGS["url"], "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(ARGS["url"], "artifacts.visualstudio.com") >= 0 ^
set additionalWgetArgs to when(isInternal, '--header="Authorization: Basic `echo $ACCESSTOKEN:$ACCESSTOKEN | base64 -w 0`" ', '') ^
set additionalCurlArgs to when(isInternal, '-u :$ACCESSTOKEN --basic ', '') ^
set isAlpine to find(OS_VERSION, "alpine") >= 0 ^
Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/Dockerfile.windows.download-file
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
sha-var-name: Name of variable that stores the checksum
hash-algorithm: Algorithm type to use to get the checksum. Defaults to sha512 ^

set isInternal to find(ARGS["url"], "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(ARGS["url"], "artifacts.visualstudio.com") >= 0 ^
set hashAlgorithm to when(ARGS["hash-algorithm"], ARGS["hash-algorithm"], "sha512")
}}{{if isInternal:`
$Base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(\":$($Env:ACCESSTOKEN)\")); `
Expand Down
19 changes: 15 additions & 4 deletions eng/dockerfile-templates/Get-GeneratedDockerfiles.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,33 @@
#!/usr/bin/env pwsh
param(
[switch]$Validate,
[string]$Branch
[string]$Branch,
[string]$OutputDirectory,
[switch]$IsInternalOverride
)

Import-Module -force $PSScriptRoot/../DependencyManagement.psm1

$customImageBuilderArgs = ""

if ($Validate) {
$customImageBuilderArgs = " --validate"
$customImageBuilderArgs += " --validate"
}

if ($IsInternalOverride) {
$customImageBuilderArgs += " --var IsInternal=true"
}

$repoRoot = (Get-Item "$PSScriptRoot").Parent.Parent.FullName
if (-Not $OutputDirectory) {
$repoRoot = (Get-Item "$PSScriptRoot").Parent.Parent.FullName
$OutputDirectory = $repoRoot
}

$onDockerfilesGenerated = {
param($ContainerName)

if (-Not $Validate) {
Exec "docker cp ${ContainerName}:/repo/src $repoRoot"
Exec "docker cp ${ContainerName}:/repo/src $OutputDirectory"
}
}

Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/aspnet/Dockerfile.linux
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
set isFullAzureLinux to isAzureLinux && !isDistroless ^
set isDistrolessAzureLinux to isAzureLinux && isDistroless ^
set baseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, I'd like to avoid having to depend on an IsInternal variable to keep things simple. Is it possible for the Get-GeneratedDockerfiles script to override the URL instead so that it matches with artifacts.visualstudio.com? Or for the test to update the content of the manifest.versions.json file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the first thing that I tried. I ran into lots and lots of issues getting the right combination of escape characters to pass arguments through with the correct --var variable="foo" syntax. It ended up wasting lots of time. I added the new override variable just to get things working. I can give this one more try though.

set isSingleStage to isAlpine && !isInternal ^
set runtimeDepsVariant to when(ARGS["is-extra"], "-extra", "") ^
set tagVersion to when(dotnetVersion = "8.0",
Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/aspnet/Dockerfile.linux-composite
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
set isFullAzureLinux to isAzureLinux && !isDistroless ^
set isDistrolessAzureLinux to isAzureLinux && isDistroless ^
set baseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isSingleStage to isAlpine && !isInternal ^
set runtimeDepsVariant to when(ARGS["is-extra"], "-extra", "") ^
set tagVersion to when(dotnetVersion = "8.0",
Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/aspnet/Dockerfile.windows
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
set isServerCore to find(OS_VERSION, "windowsservercore") >= 0 ^

set dotnetBaseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isSingleStage to (find(OS_VERSION, "windowsservercore") >= 0 && !isInternal) ^

set installerStageFromImage to cat("mcr.microsoft.com/windows/servercore:", OS_VERSION_NUMBER, "-amd64") ^
Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/runtime-deps/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
set isAzureLinux to isCblMariner || defined(match(OS_VERSION, "^azurelinux\d+\.\d+$")) ^

set baseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^

set baseImageRepo to when(isAlpine,
cat(ARCH_VERSIONED, "/alpine"),
Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/runtime/Dockerfile.linux
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
set isFullAzureLinux to isAzureLinux && !isDistroless ^
set isDistrolessAzureLinux to isAzureLinux && isDistroless ^
set baseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isSingleStage to isAlpine && !isInternal ^
set runtimeDepsVariant to when(ARGS["is-extra"], "-extra", "") ^
set tagVersion to VARIABLES[cat("dotnet|", dotnetVersion, "|fixed-tag")] ^
Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/runtime/Dockerfile.windows
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
set isServer2025 to find(OS_VERSION_NUMBER, "2025") >= 0 ^

set dotnetBaseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isSingleStage to (find(OS_VERSION, "windowsservercore") >= 0 && !isInternal) ^

set installerStageRepo to cat("mcr.microsoft.com/windows/", "servercore") ^
Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/sdk/Dockerfile.linux
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
set isMariner to find(OS_VERSION, "cbl-mariner") >= 0 ^
set isAzureLinux to isMariner || find(OS_VERSION, "azurelinux") >= 0 ^
set baseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set tagVersion to VARIABLES[cat("dotnet|", dotnetVersion, "|fixed-tag")] ^
set baseImageTag to cat("$REPO:", tagVersion, "-", OS_VERSION, ARCH_TAG_SUFFIX) ^

Expand Down
2 changes: 1 addition & 1 deletion eng/dockerfile-templates/sdk/Dockerfile.windows
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{
set dotnetVersion to join(slice(split(PRODUCT_VERSION, "."), 0, 2), ".") ^
set baseUrl to VARIABLES[cat("dotnet|", dotnetVersion, "|base-url|", VARIABLES["branch"])] ^
set isInternal to find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isInternal to VARIABLES["IsInternal"] || find(baseUrl, "artifacts.visualstudio.com") >= 0 ^
set isSingleStage to (find(OS_VERSION, "windowsservercore") >= 0 && !isInternal) ^

set installerStageFromImage to cat("mcr.microsoft.com/windows/servercore:", OS_VERSION_NUMBER, "-amd64") ^
Expand Down
35 changes: 35 additions & 0 deletions ...sOutput/aspire-dashboard/9.0/cbl-mariner-distroless/amd64/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 35 additions & 0 deletions ...utput/aspire-dashboard/9.0/cbl-mariner-distroless/arm64v8/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions ...alDockerfilesOutput/aspnet/8.0/alpine3.20-composite/amd64/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions ...DockerfilesOutput/aspnet/8.0/alpine3.20-composite/arm32v7/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions ...DockerfilesOutput/aspnet/8.0/alpine3.20-composite/arm64v8/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 23 additions & 0 deletions ...rifyInternalDockerfilesOutput/aspnet/8.0/alpine3.20/amd64/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 23 additions & 0 deletions ...fyInternalDockerfilesOutput/aspnet/8.0/alpine3.20/arm32v7/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 23 additions & 0 deletions ...fyInternalDockerfilesOutput/aspnet/8.0/alpine3.20/arm64v8/Dockerfile.verified.noextension
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading