build-and-test.ps1 -OS option not working correctly with tests

[MichaelSimons]

As part of the work to add Azure Linux 3.0 images I was trying to utilize build-and-test in my local dev experience as suggested by the new distro checklist. What I discovered is that none of the Azure Linux 3.0 specific tests are being run when I do this.

.\build-and-test.ps1 -OS azurelinux3.0

All of the distro specific tests show No data found for <Test Case Name>

Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyNoSasToken(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyNoSasToken(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SampleImageTests.VerifyDotnetSample(imageData: ???)	Microsoft.DotNet.Docker.Tests.SampleImageTests.Microsoft.DotNet.Docker.Tests.SampleImageTests.VerifyDotnetSample(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyPowerShellScenario_NonDefaultUser(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyPowerShellScenario_NonDefaultUser(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyEnvironmentVariables(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyEnvironmentVariables(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyInsecureFiles(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyInsecureFiles(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyInsecureFiles(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyInsecureFiles(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyTarInstallation(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyTarInstallation(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyDefaultUser(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyDefaultUser(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyEnvironmentVariables(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyEnvironmentVariables(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyDefaultUser(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyDefaultUser(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorNoHttpsWithAuth(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorNoHttpsWithAuth(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyInstalledPackages(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyInstalledPackages(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyBlazorWasmScenario(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyBlazorWasmScenario(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SampleImageTests.VerifyAspnetSample(imageData: ???)	Microsoft.DotNet.Docker.Tests.SampleImageTests.Microsoft.DotNet.Docker.Tests.SampleImageTests.VerifyAspnetSample(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyInsecureFiles(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyInsecureFiles(imageData: ???)		
Passed	Microsoft.DotNet.Docker.Tests.GeneratedArtifactTests.VerifyReadmeTemplates	Microsoft.DotNet.Docker.Tests.GeneratedArtifactTests.Microsoft.DotNet.Docker.Tests.GeneratedArtifactTests.VerifyReadmeTemplates		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyListenMode(imageData: ???, sampleData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyListenMode(imageData: ???, sampleData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyAotAppScenario(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyAotAppScenario(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyNoSasToken(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyNoSasToken(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyConnectMode(imageData: ???, sampleData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyConnectMode(imageData: ???, sampleData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyNoSasToken(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyNoSasToken(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyDefaultUser(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyDefaultUser(imageData: ???)		
Passed	Microsoft.DotNet.Docker.Tests.SampleImageTests.VerifyComplexAppSample	Microsoft.DotNet.Docker.Tests.SampleImageTests.Microsoft.DotNet.Docker.Tests.SampleImageTests.VerifyComplexAppSample		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyAppScenario(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyAppScenario(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyNoSasToken(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyNoSasToken(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyPackageInstallation(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyPackageInstallation(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyPowerShellScenario_DefaultUser(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyPowerShellScenario_DefaultUser(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorNoHttpsNoAuth(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorNoHttpsNoAuth(imageData: ???)		
Passed	Microsoft.DotNet.Docker.Tests.GeneratedArtifactTests.VerifyDockerfileTemplates	Microsoft.DotNet.Docker.Tests.GeneratedArtifactTests.Microsoft.DotNet.Docker.Tests.GeneratedArtifactTests.VerifyDockerfileTemplates		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyAppScenario(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyAppScenario(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyInsecureFiles(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyInsecureFiles(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyDefaultUser(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyDefaultUser(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyInstalledPackages(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyInstalledPackages(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyDefaultUser(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyDefaultUser(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyInsecureFiles(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyInsecureFiles(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyShellNotInstalledForDistroless(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorDefault(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorDefault(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorNoHttpsUnconfiguredAuth(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyMonitorNoHttpsUnconfiguredAuth(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyInstalledPackages(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyInstalledPackages(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyGitInstallation(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyGitInstallation(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyPackageInstallation(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyPackageInstallation(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyDistrolessOSReleaseInfo(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyDistrolessOSReleaseInfo(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyInstalledPackages(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyInstalledPackages(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyEnvironmentVariables(imageData: ???)	Microsoft.DotNet.Docker.Tests.MonitorImageTests.Microsoft.DotNet.Docker.Tests.MonitorImageTests.VerifyEnvironmentVariables(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyEnvironmentVariables(imageData: ???)	Microsoft.DotNet.Docker.Tests.AspnetImageTests.Microsoft.DotNet.Docker.Tests.AspnetImageTests.VerifyEnvironmentVariables(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyEnvironmentVariables(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyEnvironmentVariables(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyDotnetFolderContents(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyDotnetFolderContents(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyInstalledRpmPackages(imageData: ???)	Microsoft.DotNet.Docker.Tests.SdkImageTests.Microsoft.DotNet.Docker.Tests.SdkImageTests.VerifyInstalledRpmPackages(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyPackageInstallation(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeImageTests.Microsoft.DotNet.Docker.Tests.RuntimeImageTests.VerifyPackageInstallation(imageData: ???)		
Not Executed	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyNoSasToken(imageData: ???)	Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.Microsoft.DotNet.Docker.Tests.RuntimeDepsImageTests.VerifyNoSasToken(imageData: ???)		

[lbussell]

A workaround for this is to call the test script directly with a path argument: .\tests\run-tests.ps1 -paths "*azurelinux3.0*".