Update all non-deterministic tests to run 3x the normal CI/CD workflow on the release validation pipeline #3458
Conversation
matthewige
requested review from
dthaler,
poornagmsft,
Alan-Jowett,
saxena-anurag,
shankarseal,
dv-msft,
gtrevi,
shpalani,
mtfriesen and
rectified95
as code owners
|
All, is there any way I can test this without merging this PR? (I could not seem to schedule the release validation CI on this branch). |
To schedule, you need to have "signed-off" added under Labels in the Issue. (Ref: #3261). To test you can 1. temp. create an issue and add the label by yourself, 2. use the existing issue to add the label temp. to trigger. |
matthewige
added
signed-off
| needs: regular | ||
| if: github.event_name == 'workflow_dispatch' | ||
| uses: ./.github/workflows/reusable-test.yml | ||
| with: | ||
| name: km_mt_stress_tests | ||
| name: km_mt_stress_tests-1 | ||
| pre_test: .\setup_ebpf_cicd_tests.ps1 -KmTracing $true -KmTraceType "memory" |
There was a problem hiding this comment.
Why does this require 3 iterations here rather than making .\execute_ebpf_cicd_tests.ps1 have a parameter to handle it in there?
docs/ReleaseProcess.md
Outdated
| Once potential fixes are merged to the release branch repeat the process until the workflow completes successfully. | ||
|
|
||
| Note: If any non-deterministic tests (such as fuzzer or stress) are added in the future, the release validation workflow should be updated to run 3x the normal | ||
| CI/CD workflow (either 3x the duration or 3 instances of the test). |
There was a problem hiding this comment.
Is there any valid reason to run 3 instances of the test (as opposed to 3x the duration)? If so, what is the reason?
This reverts commit 4b20828.
Description
Release managers were instructed to run the release validation workflow 3 times to ensure non-deterministic tests had enough time/attempts to run. This change updates the release validation to take care of this, and the manager should now only need to run the workflow once.
Closes #3376
Testing
None.
Documentation
Updated the release process documentation to indicate that only 1 run is required now.
Installation
n/a